2024-09-13T09:03:20.486 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T09:03:20.503 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T09:03:20.591 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022 branch: reef description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901022' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: reef ceph: conf: client: rbd cache: true rbd cache max dirty: 0 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 248 sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 sleep_before_teardown: 0 subset: 116/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGKI0SD16larEaK7qL3fVqFA11FbcmJpsb4IcefiXnbll4p4C4mcv5bUPPSto1cOjvYegynPUinhln7rMA507Ro= smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFX1RgxW6Y1kxefH4lcBkWNrOTP/uB2pgrtM5z5hz6SpEBSwrj+qsb6iyGQeS8ha0NU38vVAH15W+Lxh34jmx2c= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_22:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T09:03:20.592 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa; will attempt to use it 2024-09-13T09:03:20.592 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa/tasks 2024-09-13T09:03:20.593 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T09:03:20.594 INFO:teuthology.task.internal:Checking packages... 2024-09-13T09:03:20.615 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '48a7de7e6e1701a88fd571c7e34908ac4b8ed412' 2024-09-13T09:03:20.616 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T09:03:20.616 INFO:teuthology.packaging:ref: None 2024-09-13T09:03:20.616 INFO:teuthology.packaging:tag: None 2024-09-13T09:03:20.616 INFO:teuthology.packaging:branch: reef 2024-09-13T09:03:20.616 INFO:teuthology.packaging:sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:03:20.616 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-09-13T09:03:20.875 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-843-g2c01458a-1jammy 2024-09-13T09:03:20.877 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T09:03:20.882 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T09:03:20.882 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T09:03:20.888 INFO:teuthology.task.internal:Saving configuration 2024-09-13T09:03:20.900 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T09:03:20.909 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T09:03:20.932 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 08:57:33.802290', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJ/eK3yUutJ2GEnxj39kqhmk2OjKn5ZWflNKjbsQRkD9'} 2024-09-13T09:03:20.954 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 08:57:33.803480', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKb7xzmBUp03+LWelyh7fVpTUUokJ6meGzV4RvHGtCnZ'} 2024-09-13T09:03:20.954 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T09:03:20.960 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-13T09:03:20.960 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-13T09:03:20.960 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T09:03:21.023 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1b0114f640>, signals=[15]) 2024-09-13T09:03:21.023 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T09:03:21.031 INFO:teuthology.task.internal:Opening connections... 2024-09-13T09:03:21.031 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2024-09-13T09:03:21.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:03:21.113 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-09-13T09:03:21.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:03:21.196 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T09:03:21.203 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2024-09-13T09:03:21.209 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2024-09-13T09:03:21.209 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-09-13T09:03:21.256 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T09:03:21.256 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Ubuntu" 2024-09-13T09:03:21.256 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="22.04" 2024-09-13T09:03:21.256 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T09:03:21.256 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_CODENAME=jammy 2024-09-13T09:03:21.256 INFO:teuthology.orchestra.run.smithi032.stdout:ID=ubuntu 2024-09-13T09:03:21.256 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE=debian 2024-09-13T09:03:21.257 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T09:03:21.257 INFO:teuthology.orchestra.run.smithi032.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T09:03:21.257 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T09:03:21.257 INFO:teuthology.orchestra.run.smithi032.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T09:03:21.257 INFO:teuthology.orchestra.run.smithi032.stdout:UBUNTU_CODENAME=jammy 2024-09-13T09:03:21.257 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-09-13T09:03:21.277 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-09-13T09:03:21.282 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-09-13T09:03:21.283 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-09-13T09:03:21.329 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T09:03:21.329 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Ubuntu" 2024-09-13T09:03:21.330 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="22.04" 2024-09-13T09:03:21.330 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T09:03:21.330 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_CODENAME=jammy 2024-09-13T09:03:21.330 INFO:teuthology.orchestra.run.smithi046.stdout:ID=ubuntu 2024-09-13T09:03:21.330 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE=debian 2024-09-13T09:03:21.330 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T09:03:21.330 INFO:teuthology.orchestra.run.smithi046.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T09:03:21.330 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T09:03:21.330 INFO:teuthology.orchestra.run.smithi046.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T09:03:21.330 INFO:teuthology.orchestra.run.smithi046.stdout:UBUNTU_CODENAME=jammy 2024-09-13T09:03:21.331 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-09-13T09:03:21.352 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T09:03:21.361 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T09:03:21.367 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T09:03:21.368 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2024-09-13T09:03:21.370 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-09-13T09:03:21.379 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T09:03:21.386 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T09:03:21.386 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2024-09-13T09:03:21.417 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-09-13T09:03:21.432 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T09:03:21.480 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T09:03:21.494 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T09:03:21.494 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T09:03:21.494 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T09:03:21.495 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T09:03:21.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:03:21.499 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-09-13T09:03:21.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:03:21.500 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-13T09:03:21.548 INFO:teuthology.orchestra.run.smithi032.stdout:5.15.0-56-generic 2024-09-13T09:03:21.548 INFO:teuthology.task.kernel:Running kernel on smithi032: 5.15.0-56-generic 2024-09-13T09:03:21.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get clean 2024-09-13T09:03:21.549 INFO:teuthology.orchestra.run.smithi046.stdout:5.15.0-56-generic 2024-09-13T09:03:21.550 INFO:teuthology.task.kernel:Running kernel on smithi046: 5.15.0-56-generic 2024-09-13T09:03:21.550 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get clean 2024-09-13T09:03:21.671 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2024-09-13T09:03:21.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2024-09-13T09:03:21.875 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T09:03:21.962 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:03:21.962 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T09:03:21.982 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:03:22.008 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T09:03:22.064 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T09:03:22.094 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T09:03:22.166 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-13T09:03:22.199 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T09:03:22.202 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T09:03:22.202 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T09:03:22.203 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T09:03:22.225 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T09:03:22.229 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T09:03:22.229 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T09:03:22.235 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T09:03:22.243 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T09:03:22.244 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T09:03:22.244 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T09:03:22.245 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T09:03:22.245 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T09:03:22.253 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T09:03:22.363 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T09:03:22.389 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-13T09:03:22.418 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T09:03:22.501 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T09:03:22.556 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-13T09:03:22.703 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-13T09:03:22.808 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T09:03:22.825 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T09:03:22.833 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T09:03:22.834 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T09:03:22.835 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T09:03:22.853 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T09:03:22.867 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T09:03:22.868 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T09:03:22.869 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T09:03:22.873 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T09:03:22.908 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T09:03:22.922 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T09:03:22.923 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T09:03:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T09:03:22.938 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T09:03:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T09:03:22.972 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T09:03:22.973 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T09:03:22.973 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T09:03:22.984 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T09:03:22.986 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T09:03:22.992 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T09:03:22.993 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T09:03:23.000 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T09:03:23.004 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T09:03:23.004 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T09:03:23.005 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T09:03:23.006 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T09:03:23.007 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T09:03:23.009 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T09:03:23.009 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T09:03:23.011 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T09:03:23.017 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T09:03:23.018 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T09:03:23.019 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T09:03:23.031 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T09:03:23.060 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T09:03:23.066 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T09:03:23.067 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T09:03:23.068 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T09:03:23.074 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T09:03:23.093 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T09:03:23.094 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T09:03:23.096 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T09:03:23.097 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T09:03:23.097 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T09:03:23.099 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T09:03:23.099 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T09:03:23.117 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T09:03:23.117 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T09:03:23.118 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T09:03:23.119 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T09:03:23.121 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T09:03:23.180 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T09:03:23.181 INFO:teuthology.orchestra.run.smithi046.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T09:03:23.198 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T09:03:23.198 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T09:03:23.199 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T09:03:23.200 INFO:teuthology.orchestra.run.smithi032.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T09:03:23.201 INFO:teuthology.orchestra.run.smithi032.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T09:03:28.809 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 16.0 MB in 2s (7,238 kB/s) 2024-09-13T09:03:29.518 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 16.0 MB in 3s (5,699 kB/s) 2024-09-13T09:03:30.389 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-09-13T09:03:30.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:03:30.473 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-09-13T09:03:30.688 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2024-09-13T09:03:30.688 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2024-09-13T09:03:30.886 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:03:30.887 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:03:30.887 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:03:30.887 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:03:30.887 INFO:teuthology.orchestra.run.smithi032.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T09:03:30.887 INFO:teuthology.orchestra.run.smithi032.stdout: python2.7-minimal 2024-09-13T09:03:30.888 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:03:30.889 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2024-09-13T09:03:30.889 INFO:teuthology.orchestra.run.smithi032.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T09:03:30.889 INFO:teuthology.orchestra.run.smithi032.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T09:03:30.889 INFO:teuthology.orchestra.run.smithi032.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T09:03:30.891 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2024-09-13T09:03:30.891 INFO:teuthology.orchestra.run.smithi032.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T09:03:30.891 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2024-09-13T09:03:30.891 INFO:teuthology.orchestra.run.smithi032.stdout: thermald 2024-09-13T09:03:30.946 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2024-09-13T09:03:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T09:03:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T09:03:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T09:03:30.948 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2024-09-13T09:03:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T09:03:31.002 INFO:teuthology.orchestra.run.smithi032.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T09:03:31.003 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 113 MB of archives. 2024-09-13T09:03:31.003 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T09:03:31.003 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T09:03:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-13T09:03:31.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:03:31.082 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-13T09:03:31.280 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2024-09-13T09:03:31.281 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2024-09-13T09:03:31.434 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T09:03:31.455 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:03:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:03:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:03:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:03:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T09:03:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: python2.7-minimal 2024-09-13T09:03:31.456 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:03:31.458 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2024-09-13T09:03:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T09:03:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T09:03:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T09:03:31.459 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2024-09-13T09:03:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T09:03:31.459 INFO:teuthology.orchestra.run.smithi046.stdout:Recommended packages: 2024-09-13T09:03:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: thermald 2024-09-13T09:03:31.505 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2024-09-13T09:03:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T09:03:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T09:03:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T09:03:31.507 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages will be upgraded: 2024-09-13T09:03:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T09:03:31.531 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T09:03:31.687 INFO:teuthology.orchestra.run.smithi046.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T09:03:31.687 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 113 MB of archives. 2024-09-13T09:03:31.687 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T09:03:31.687 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T09:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T09:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T09:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T09:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T09:03:32.505 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T09:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 113 MB in 2s (69.9 MB/s) 2024-09-13T09:03:33.068 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T09:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T09:03:33.425 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T09:03:35.406 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T09:03:35.411 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:03:35.639 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T09:03:35.639 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T09:03:35.639 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T09:03:35.670 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:03:35.982 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T09:03:36.060 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T09:03:36.476 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 113 MB in 5s (24.9 MB/s) 2024-09-13T09:03:36.674 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T09:03:37.914 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T09:03:37.935 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:03:38.020 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:03:38.534 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T09:03:38.554 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:03:38.592 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:03:38.668 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T09:03:38.673 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:03:38.773 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:03:41.465 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T09:03:41.486 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:03:41.573 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:03:42.001 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T09:03:42.022 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:03:42.059 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:03:46.018 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:03:46.183 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:03:46.524 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:03:46.678 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:03:46.998 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T09:03:47.012 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T09:03:47.065 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T09:03:48.657 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:03:48.817 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:03:49.151 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:03:49.271 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:03:49.558 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T09:03:49.580 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T09:03:49.623 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T09:03:53.078 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T09:03:53.115 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:03:53.161 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:03:55.301 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T09:03:55.339 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:03:55.376 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:03:56.643 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:03:56.780 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:03:57.139 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T09:03:57.660 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:03:58.429 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T09:03:58.557 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:03:59.697 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:03:59.819 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:03:59.914 INFO:teuthology.orchestra.run.smithi032.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:03:59.914 INFO:teuthology.orchestra.run.smithi032.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T09:03:59.914 INFO:teuthology.orchestra.run.smithi032.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:03:59.914 INFO:teuthology.orchestra.run.smithi032.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T09:04:00.084 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:04:00.186 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T09:04:00.332 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:04:00.441 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T09:04:00.552 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:04:01.560 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:04:01.877 INFO:teuthology.orchestra.run.smithi046.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:04:01.878 INFO:teuthology.orchestra.run.smithi046.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T09:04:01.878 INFO:teuthology.orchestra.run.smithi046.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:04:01.878 INFO:teuthology.orchestra.run.smithi046.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T09:04:02.072 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:04:02.978 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T09:04:03.129 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T09:04:03.281 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:04:03.343 INFO:teuthology.orchestra.run.smithi032.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T09:04:03.344 INFO:teuthology.orchestra.run.smithi032.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:04:04.273 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:04:05.774 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T09:04:05.908 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T09:04:06.026 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:04:06.094 INFO:teuthology.orchestra.run.smithi046.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T09:04:06.094 INFO:teuthology.orchestra.run.smithi046.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:04:18.416 INFO:teuthology.orchestra.run.smithi032.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T09:04:18.416 INFO:teuthology.orchestra.run.smithi032.stdout:Sourcing file `/etc/default/grub' 2024-09-13T09:04:18.439 INFO:teuthology.orchestra.run.smithi032.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:04:18.455 INFO:teuthology.orchestra.run.smithi032.stdout:Generating grub configuration file ... 2024-09-13T09:04:18.797 INFO:teuthology.orchestra.run.smithi032.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:04:18.816 INFO:teuthology.orchestra.run.smithi032.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:04:19.165 INFO:teuthology.orchestra.run.smithi032.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:04:19.171 INFO:teuthology.orchestra.run.smithi032.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T09:04:19.245 INFO:teuthology.orchestra.run.smithi032.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T09:04:19.251 INFO:teuthology.orchestra.run.smithi032.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T09:04:19.433 INFO:teuthology.orchestra.run.smithi032.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T09:04:19.433 INFO:teuthology.orchestra.run.smithi032.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T09:04:19.433 INFO:teuthology.orchestra.run.smithi032.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T09:04:19.490 INFO:teuthology.orchestra.run.smithi032.stdout:done 2024-09-13T09:04:20.045 DEBUG:teuthology.orchestra.run.smithi032:> dpkg -s linux-image-generic 2024-09-13T09:04:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:Package: linux-image-generic 2024-09-13T09:04:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:Status: install ok installed 2024-09-13T09:04:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:Priority: optional 2024-09-13T09:04:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:Section: kernel 2024-09-13T09:04:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:Installed-Size: 21 2024-09-13T09:04:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:04:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: amd64 2024-09-13T09:04:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:Source: linux-meta 2024-09-13T09:04:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:Version: 5.15.0.121.121 2024-09-13T09:04:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T09:04:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:04:20.075 INFO:teuthology.orchestra.run.smithi032.stdout:Recommends: thermald 2024-09-13T09:04:20.075 INFO:teuthology.orchestra.run.smithi032.stdout:Description: Generic Linux kernel image 2024-09-13T09:04:20.075 INFO:teuthology.orchestra.run.smithi032.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:04:20.075 INFO:teuthology.orchestra.run.smithi032.stdout: available. 2024-09-13T09:04:20.075 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T09:04:20.075 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T09:04:20.075 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T09:04:20.075 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T09:04:20.076 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi032.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T09:04:20.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get clean 2024-09-13T09:04:20.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2024-09-13T09:04:20.441 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T09:04:20.562 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:04:20.652 INFO:teuthology.orchestra.run.smithi046.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T09:04:20.652 INFO:teuthology.orchestra.run.smithi046.stdout:Sourcing file `/etc/default/grub' 2024-09-13T09:04:20.657 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T09:04:20.677 INFO:teuthology.orchestra.run.smithi046.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:04:20.696 INFO:teuthology.orchestra.run.smithi046.stdout:Generating grub configuration file ... 2024-09-13T09:04:20.724 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T09:04:21.030 INFO:teuthology.orchestra.run.smithi046.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:04:21.056 INFO:teuthology.orchestra.run.smithi046.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:04:21.472 INFO:teuthology.orchestra.run.smithi046.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:04:21.479 INFO:teuthology.orchestra.run.smithi046.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T09:04:21.548 INFO:teuthology.orchestra.run.smithi046.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T09:04:21.554 INFO:teuthology.orchestra.run.smithi046.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T09:04:21.746 INFO:teuthology.orchestra.run.smithi046.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T09:04:21.746 INFO:teuthology.orchestra.run.smithi046.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T09:04:21.746 INFO:teuthology.orchestra.run.smithi046.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T09:04:21.784 INFO:teuthology.orchestra.run.smithi046.stdout:done 2024-09-13T09:04:21.916 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-09-13T09:04:21.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:04:22.002 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-09-13T09:04:22.221 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2024-09-13T09:04:22.222 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2024-09-13T09:04:22.283 DEBUG:teuthology.orchestra.run.smithi046:> dpkg -s linux-image-generic 2024-09-13T09:04:22.341 INFO:teuthology.orchestra.run.smithi046.stdout:Package: linux-image-generic 2024-09-13T09:04:22.341 INFO:teuthology.orchestra.run.smithi046.stdout:Status: install ok installed 2024-09-13T09:04:22.341 INFO:teuthology.orchestra.run.smithi046.stdout:Priority: optional 2024-09-13T09:04:22.341 INFO:teuthology.orchestra.run.smithi046.stdout:Section: kernel 2024-09-13T09:04:22.341 INFO:teuthology.orchestra.run.smithi046.stdout:Installed-Size: 21 2024-09-13T09:04:22.341 INFO:teuthology.orchestra.run.smithi046.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:04:22.341 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: amd64 2024-09-13T09:04:22.341 INFO:teuthology.orchestra.run.smithi046.stdout:Source: linux-meta 2024-09-13T09:04:22.341 INFO:teuthology.orchestra.run.smithi046.stdout:Version: 5.15.0.121.121 2024-09-13T09:04:22.341 INFO:teuthology.orchestra.run.smithi046.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T09:04:22.341 INFO:teuthology.orchestra.run.smithi046.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:04:22.342 INFO:teuthology.orchestra.run.smithi046.stdout:Recommends: thermald 2024-09-13T09:04:22.342 INFO:teuthology.orchestra.run.smithi046.stdout:Description: Generic Linux kernel image 2024-09-13T09:04:22.342 INFO:teuthology.orchestra.run.smithi046.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:04:22.342 INFO:teuthology.orchestra.run.smithi046.stdout: available. 2024-09-13T09:04:22.342 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T09:04:22.342 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T09:04:22.342 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T09:04:22.342 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T09:04:22.342 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T09:04:22.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get clean 2024-09-13T09:04:22.421 INFO:teuthology.orchestra.run.smithi032.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T09:04:22.421 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:04:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:04:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:04:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:04:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T09:04:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T09:04:22.422 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:04:22.467 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T09:04:22.469 DEBUG:teuthology.orchestra.run.smithi032:> dpkg -s linux-image-generic 2024-09-13T09:04:22.485 INFO:teuthology.orchestra.run.smithi032.stdout:Package: linux-image-generic 2024-09-13T09:04:22.485 INFO:teuthology.orchestra.run.smithi032.stdout:Status: install ok installed 2024-09-13T09:04:22.485 INFO:teuthology.orchestra.run.smithi032.stdout:Priority: optional 2024-09-13T09:04:22.485 INFO:teuthology.orchestra.run.smithi032.stdout:Section: kernel 2024-09-13T09:04:22.485 INFO:teuthology.orchestra.run.smithi032.stdout:Installed-Size: 21 2024-09-13T09:04:22.485 INFO:teuthology.orchestra.run.smithi032.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:04:22.485 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: amd64 2024-09-13T09:04:22.486 INFO:teuthology.orchestra.run.smithi032.stdout:Source: linux-meta 2024-09-13T09:04:22.486 INFO:teuthology.orchestra.run.smithi032.stdout:Version: 5.15.0.121.121 2024-09-13T09:04:22.486 INFO:teuthology.orchestra.run.smithi032.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T09:04:22.486 INFO:teuthology.orchestra.run.smithi032.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:04:22.486 INFO:teuthology.orchestra.run.smithi032.stdout:Recommends: thermald 2024-09-13T09:04:22.486 INFO:teuthology.orchestra.run.smithi032.stdout:Description: Generic Linux kernel image 2024-09-13T09:04:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:04:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: available. 2024-09-13T09:04:22.486 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-09-13T09:04:22.506 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2024-09-13T09:04:22.536 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.xAvwvm3TZT 2024-09-13T09:04:22.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /boot/grub/grub.cfg /tmp/tmp.xAvwvm3TZT 2024-09-13T09:04:22.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.xAvwvm3TZT 2024-09-13T09:04:22.626 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T09:04:22.706 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.xAvwvm3TZT is 10KB 2024-09-13T09:04:22.720 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.xAvwvm3TZT 2024-09-13T09:04:22.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T09:04:22.754 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:04:22.779 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:04:22.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T09:04:22.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T09:04:22.828 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T09:04:22.874 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T09:04:22.874 DEBUG:teuthology.orchestra.run.smithi032:> sudo update-grub 2024-09-13T09:04:22.910 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T09:04:24.144 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-13T09:04:24.163 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:04:24.228 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-13T09:04:24.398 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2024-09-13T09:04:24.399 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:04:24.406 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2024-09-13T09:04:24.451 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2024-09-13T09:04:24.451 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2024-09-13T09:04:24.655 INFO:teuthology.orchestra.run.smithi046.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T09:04:24.655 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:04:24.655 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:04:24.655 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:04:24.655 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:04:24.656 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T09:04:24.656 INFO:teuthology.orchestra.run.smithi046.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T09:04:24.656 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:04:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:04:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:04:24.702 INFO:teuthology.orchestra.run.smithi046.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T09:04:24.704 DEBUG:teuthology.orchestra.run.smithi046:> dpkg -s linux-image-generic 2024-09-13T09:04:24.719 INFO:teuthology.orchestra.run.smithi046.stdout:Package: linux-image-generic 2024-09-13T09:04:24.720 INFO:teuthology.orchestra.run.smithi046.stdout:Status: install ok installed 2024-09-13T09:04:24.720 INFO:teuthology.orchestra.run.smithi046.stdout:Priority: optional 2024-09-13T09:04:24.720 INFO:teuthology.orchestra.run.smithi046.stdout:Section: kernel 2024-09-13T09:04:24.720 INFO:teuthology.orchestra.run.smithi046.stdout:Installed-Size: 21 2024-09-13T09:04:24.720 INFO:teuthology.orchestra.run.smithi046.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:04:24.720 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: amd64 2024-09-13T09:04:24.720 INFO:teuthology.orchestra.run.smithi046.stdout:Source: linux-meta 2024-09-13T09:04:24.720 INFO:teuthology.orchestra.run.smithi046.stdout:Version: 5.15.0.121.121 2024-09-13T09:04:24.720 INFO:teuthology.orchestra.run.smithi046.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T09:04:24.720 INFO:teuthology.orchestra.run.smithi046.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:04:24.720 INFO:teuthology.orchestra.run.smithi046.stdout:Recommends: thermald 2024-09-13T09:04:24.721 INFO:teuthology.orchestra.run.smithi046.stdout:Description: Generic Linux kernel image 2024-09-13T09:04:24.721 INFO:teuthology.orchestra.run.smithi046.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:04:24.721 INFO:teuthology.orchestra.run.smithi046.stdout: available. 2024-09-13T09:04:24.721 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-09-13T09:04:24.768 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.lB2pgrc4ty 2024-09-13T09:04:24.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub/grub.cfg /tmp/tmp.lB2pgrc4ty 2024-09-13T09:04:24.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.lB2pgrc4ty 2024-09-13T09:04:24.933 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:04:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T09:04:24.946 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.lB2pgrc4ty is 10KB 2024-09-13T09:04:24.959 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.lB2pgrc4ty 2024-09-13T09:04:24.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T09:04:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T09:04:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T09:04:25.020 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:04:25.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T09:04:25.021 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T09:04:25.086 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T09:04:25.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo update-grub 2024-09-13T09:04:25.104 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T09:04:25.104 INFO:teuthology.orchestra.run.smithi032.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T09:04:25.104 INFO:teuthology.orchestra.run.smithi032.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T09:04:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:done 2024-09-13T09:04:25.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo shutdown -r now 2024-09-13T09:04:26.403 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub' 2024-09-13T09:04:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:04:26.408 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-09-13T09:04:26.580 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:04:26.586 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:04:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:04:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T09:04:26.980 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T09:04:26.986 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T09:04:27.158 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T09:04:27.158 INFO:teuthology.orchestra.run.smithi046.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T09:04:27.158 INFO:teuthology.orchestra.run.smithi046.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T09:04:27.174 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-09-13T09:04:27.185 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-09-13T09:04:55.216 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T09:04:55.217 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-09-13T09:04:55.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:04:57.189 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T09:04:57.189 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-13T09:04:57.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:05:13.538 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-09-13T09:05:15.585 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-13T09:05:22.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-09-13T09:05:22.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:05:24.588 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-13T09:05:24.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:05:27.645 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-13T09:05:39.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-13T09:05:39.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:05:40.925 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-09-13T09:05:52.934 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-09-13T09:05:52.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:05:55.997 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-09-13T09:06:11.012 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-09-13T09:06:11.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:06:11.310 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-09-13T09:06:11.863 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-09-13T09:06:11.864 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-13T09:06:11.864 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-09-13T09:06:11.910 INFO:teuthology.orchestra.run.smithi032.stdout:5.15.0-121-generic 2024-09-13T09:06:11.910 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T09:06:11.910 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T09:06:11.911 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T09:06:12.912 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T09:06:12.912 DEBUG:teuthology.orchestra.run.smithi032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T09:06:12.985 INFO:teuthology.orchestra.run.smithi032.stdout:ttyS1 2024-09-13T09:06:13.024 DEBUG:teuthology.parallel:result is None 2024-09-13T09:06:39.661 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T09:06:54.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-13T09:06:54.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:06:54.915 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-13T09:06:55.562 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-13T09:06:55.562 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-121-generic"... 2024-09-13T09:06:55.562 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-13T09:06:55.610 INFO:teuthology.orchestra.run.smithi046.stdout:5.15.0-121-generic 2024-09-13T09:06:55.611 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T09:06:55.611 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T09:06:55.611 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T09:06:56.613 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T09:06:56.614 DEBUG:teuthology.orchestra.run.smithi046:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T09:06:56.677 INFO:teuthology.orchestra.run.smithi046.stdout:ttyS1 2024-09-13T09:06:56.696 DEBUG:teuthology.parallel:result is None 2024-09-13T09:06:56.696 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T09:06:56.703 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T09:06:56.703 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T09:06:56.706 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T09:06:56.711 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T09:06:56.718 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T09:06:56.729 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T09:06:56.729 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T09:06:56.755 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T09:06:56.771 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T09:06:56.779 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T09:06:56.779 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T09:06:56.811 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T09:06:56.826 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:06:56.834 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:06:56.835 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:06:56.843 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:06:56.844 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T09:06:56.851 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T09:06:56.851 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T09:06:56.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T09:06:56.908 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T09:06:56.918 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T09:06:56.918 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T09:06:56.935 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T09:06:56.955 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:06:56.986 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:06:57.034 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:06:57.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T09:06:57.096 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:06:57.102 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:06:57.151 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:06:57.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T09:06:57.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2024-09-13T09:06:57.217 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-09-13T09:06:57.293 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T09:06:57.301 INFO:teuthology.task.internal:Starting timer... 2024-09-13T09:06:57.301 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T09:06:57.311 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T09:06:57.319 DEBUG:teuthology.task.selinux:Excluding smithi032: OS 'ubuntu' does not support SELinux 2024-09-13T09:06:57.320 DEBUG:teuthology.task.selinux:Excluding smithi046: OS 'ubuntu' does not support SELinux 2024-09-13T09:06:57.320 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T09:06:57.320 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T09:06:57.320 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T09:06:57.320 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T09:06:57.328 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T09:06:57.340 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T09:06:57.341 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi046.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T09:14:34.045 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi046.front.sepia.ceph.com')] 2024-09-13T09:14:34.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-09-13T09:14:34.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:14:34.135 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-09-13T09:14:34.213 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-09-13T09:14:34.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-13T09:14:34.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:14:34.298 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-13T09:14:34.377 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-13T09:14:34.378 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T09:14:34.393 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T09:14:34.393 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T09:14:34.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:14:34.396 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T09:14:34.396 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:14:34.423 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T09:14:34.423 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: Command line: ntpd -gq 2024-09-13T09:14:34.423 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: ---------------------------------------------------- 2024-09-13T09:14:34.423 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T09:14:34.423 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T09:14:34.424 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: corporation. Support and training for ntp-4 are 2024-09-13T09:14:34.424 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: available at https://www.nwtime.org/support 2024-09-13T09:14:34.424 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: ---------------------------------------------------- 2024-09-13T09:14:34.425 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: proto: precision = 0.045 usec (-24) 2024-09-13T09:14:34.425 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: basedate set to 2022-02-04 2024-09-13T09:14:34.425 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: gps base set to 2022-02-06 (week 2196) 2024-09-13T09:14:34.425 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T09:14:34.425 INFO:teuthology.orchestra.run.smithi032.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T09:14:34.426 INFO:teuthology.orchestra.run.smithi032.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T09:14:34.426 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T09:14:34.427 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T09:14:34.427 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T09:14:34.427 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T09:14:34.427 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: Listen normally on 3 ens1f0 172.21.15.32:123 2024-09-13T09:14:34.427 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: Listen normally on 4 lo [::1]:123 2024-09-13T09:14:34.427 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:2602%4]:123 2024-09-13T09:14:34.427 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:34 ntpd[18355]: Listening on routing socket on fd #22 for interface updates 2024-09-13T09:14:34.450 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T09:14:34.450 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: Command line: ntpd -gq 2024-09-13T09:14:34.450 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: ---------------------------------------------------- 2024-09-13T09:14:34.450 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T09:14:34.450 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T09:14:34.450 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: corporation. Support and training for ntp-4 are 2024-09-13T09:14:34.450 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: available at https://www.nwtime.org/support 2024-09-13T09:14:34.450 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: ---------------------------------------------------- 2024-09-13T09:14:34.450 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: proto: precision = 0.046 usec (-24) 2024-09-13T09:14:34.450 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: basedate set to 2022-02-04 2024-09-13T09:14:34.450 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: gps base set to 2022-02-06 (week 2196) 2024-09-13T09:14:34.451 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T09:14:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T09:14:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T09:14:34.452 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T09:14:34.452 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T09:14:34.452 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T09:14:34.452 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T09:14:34.452 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: Listen normally on 3 ens1f0 172.21.15.46:123 2024-09-13T09:14:34.452 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: Listen normally on 4 lo [::1]:123 2024-09-13T09:14:34.452 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15f8%4]:123 2024-09-13T09:14:34.453 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:34 ntpd[18385]: Listening on routing socket on fd #22 for interface updates 2024-09-13T09:14:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:35 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:35.423 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:35 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:35.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:35 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:35.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:35 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:35.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:35 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:35.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:35 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:35.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:35 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:35.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:35 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:35.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:35 ntpd[18385]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:35.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:35 ntpd[18385]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:36 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:36 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:36 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:36 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:36 ntpd[18355]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:36 ntpd[18355]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:37 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:37 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:37 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:37 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:37 ntpd[18355]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:37 ntpd[18355]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:37.449 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:37 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:37.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:37 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:37.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:37 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:37.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:37 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:37.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:37 ntpd[18385]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:37.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:37 ntpd[18385]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:38.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:38 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:38.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:38 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:38.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:38 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:38.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:38 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:38.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:38 ntpd[18355]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:38.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:38 ntpd[18355]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:38.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:38 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:38.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:38 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:38.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:38 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:38.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:38 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:38.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:38 ntpd[18385]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:38.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:38 ntpd[18385]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:39 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:39 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:39 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:39 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:39 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:39 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:39 ntpd[18355]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:39 ntpd[18355]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:39 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:39 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:39 ntpd[18355]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:39 ntpd[18355]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:39 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:39 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:39 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:39 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:39 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:39 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:39 ntpd[18385]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:39 ntpd[18385]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:39 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:39 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:39 ntpd[18385]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:39 ntpd[18385]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:40 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:40 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:40 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:40 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:40 ntpd[18355]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:40 ntpd[18355]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:40.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:40 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:40.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:40 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:40.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:40 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:40.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:40 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:40.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:40 ntpd[18385]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:40.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:40 ntpd[18385]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:41 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:41 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:41 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:41 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:41 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:41 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:41 ntpd[18355]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:41 ntpd[18355]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:41 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:41 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:41 ntpd[18355]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:41 ntpd[18355]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:41.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:41 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:41.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:41 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:41.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:41 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:41.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:41 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:41.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:41 ntpd[18385]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:41.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:41 ntpd[18385]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:41.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:41 ntpd[18385]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:41.450 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:41 ntpd[18385]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:41.451 INFO:teuthology.orchestra.run.smithi046.stdout:13 Sep 09:14:41 ntpd[18385]: ntpd: time slew -0.000296 s 2024-09-13T09:14:41.451 INFO:teuthology.orchestra.run.smithi046.stdout:ntpd: time slew -0.000296s 2024-09-13T09:14:41.451 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:41 ntpd[18385]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T09:14:41.451 INFO:teuthology.orchestra.run.smithi046.stderr:13 Sep 09:14:41 ntpd[18385]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T09:14:41.505 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T09:14:41.505 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2024-09-13T09:14:41.505 INFO:teuthology.orchestra.run.smithi046.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:14:41.505 INFO:teuthology.orchestra.run.smithi046.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:14:41.505 INFO:teuthology.orchestra.run.smithi046.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:14:41.505 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:14:42.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:42 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:42.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:42 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:42.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:42 ntpd[18355]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:14:42.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:42 ntpd[18355]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:14:42.424 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:42 ntpd[18355]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:14:42.424 INFO:teuthology.orchestra.run.smithi032.stdout:13 Sep 09:14:42 ntpd[18355]: ntpd: time slew -0.000629 s 2024-09-13T09:14:42.425 INFO:teuthology.orchestra.run.smithi032.stdout:ntpd: time slew -0.000629s 2024-09-13T09:14:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:42 ntpd[18355]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:14:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:42 ntpd[18355]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T09:14:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:13 Sep 09:14:42 ntpd[18355]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T09:14:42.476 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T09:14:42.476 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2024-09-13T09:14:42.476 INFO:teuthology.orchestra.run.smithi032.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:14:42.476 INFO:teuthology.orchestra.run.smithi032.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:14:42.476 INFO:teuthology.orchestra.run.smithi032.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:14:42.476 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:14:42.477 INFO:teuthology.run_tasks:Running task install... 2024-09-13T09:14:42.485 DEBUG:teuthology.task.install:project ceph 2024-09-13T09:14:42.485 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}} 2024-09-13T09:14:42.485 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'} 2024-09-13T09:14:42.485 INFO:teuthology.task.install:Using flavor: default 2024-09-13T09:14:42.491 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T09:14:42.492 INFO:teuthology.task.install:extra packages: [] 2024-09-13T09:14:42.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-key list | grep Ceph 2024-09-13T09:14:42.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-key list | grep Ceph 2024-09-13T09:14:42.560 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T09:14:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T09:14:42.589 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T09:14:42.589 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T09:14:42.590 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T09:14:42.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:14:42.616 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T09:14:42.616 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T09:14:42.616 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T09:14:42.616 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:14:42.741 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default/ 2024-09-13T09:14:42.741 INFO:teuthology.task.install.deb:Package version is 18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:14:42.760 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default/ 2024-09-13T09:14:42.760 INFO:teuthology.task.install.deb:Package version is 18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:14:42.850 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:14:42.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T09:14:42.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2024-09-13T09:14:42.869 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:14:42.869 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T09:14:42.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2024-09-13T09:14:43.041 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T09:14:43.059 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:14:43.069 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T09:14:43.086 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T09:14:43.095 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T09:14:43.115 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T09:14:43.118 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-13T09:14:43.136 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-13T09:14:43.139 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T09:14:43.159 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T09:14:43.160 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,421 B] 2024-09-13T09:14:43.179 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:14:43.180 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-09-13T09:14:43.185 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T09:14:43.189 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-09-13T09:14:43.207 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,421 B] 2024-09-13T09:14:43.260 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T09:14:43.342 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T09:14:43.419 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 28.8 kB in 0s (82.2 kB/s) 2024-09-13T09:14:43.494 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 28.8 kB in 0s (65.7 kB/s) 2024-09-13T09:14:44.459 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-09-13T09:14:44.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-820-g48a7de7e-1jammy cephadm=18.2.4-820-g48a7de7e-1jammy ceph-mds=18.2.4-820-g48a7de7e-1jammy ceph-mgr=18.2.4-820-g48a7de7e-1jammy ceph-common=18.2.4-820-g48a7de7e-1jammy ceph-fuse=18.2.4-820-g48a7de7e-1jammy ceph-test=18.2.4-820-g48a7de7e-1jammy ceph-volume=18.2.4-820-g48a7de7e-1jammy radosgw=18.2.4-820-g48a7de7e-1jammy python3-rados=18.2.4-820-g48a7de7e-1jammy python3-rgw=18.2.4-820-g48a7de7e-1jammy python3-cephfs=18.2.4-820-g48a7de7e-1jammy python3-rbd=18.2.4-820-g48a7de7e-1jammy libcephfs2=18.2.4-820-g48a7de7e-1jammy libcephfs-dev=18.2.4-820-g48a7de7e-1jammy librados2=18.2.4-820-g48a7de7e-1jammy librbd1=18.2.4-820-g48a7de7e-1jammy rbd-fuse=18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:14:44.531 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-13T09:14:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-09-13T09:14:44.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-820-g48a7de7e-1jammy cephadm=18.2.4-820-g48a7de7e-1jammy ceph-mds=18.2.4-820-g48a7de7e-1jammy ceph-mgr=18.2.4-820-g48a7de7e-1jammy ceph-common=18.2.4-820-g48a7de7e-1jammy ceph-fuse=18.2.4-820-g48a7de7e-1jammy ceph-test=18.2.4-820-g48a7de7e-1jammy ceph-volume=18.2.4-820-g48a7de7e-1jammy radosgw=18.2.4-820-g48a7de7e-1jammy python3-rados=18.2.4-820-g48a7de7e-1jammy python3-rgw=18.2.4-820-g48a7de7e-1jammy python3-cephfs=18.2.4-820-g48a7de7e-1jammy python3-rbd=18.2.4-820-g48a7de7e-1jammy libcephfs2=18.2.4-820-g48a7de7e-1jammy libcephfs-dev=18.2.4-820-g48a7de7e-1jammy librados2=18.2.4-820-g48a7de7e-1jammy librbd1=18.2.4-820-g48a7de7e-1jammy rbd-fuse=18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:14:44.614 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-13T09:14:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2024-09-13T09:14:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2024-09-13T09:14:44.828 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2024-09-13T09:14:44.829 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2024-09-13T09:14:44.969 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:14:44.969 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:14:44.970 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:14:44.970 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:14:44.970 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T09:14:44.970 INFO:teuthology.orchestra.run.smithi032.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T09:14:44.970 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:14:44.971 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2024-09-13T09:14:44.971 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T09:14:44.971 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T09:14:44.972 INFO:teuthology.orchestra.run.smithi032.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-13T09:14:44.972 INFO:teuthology.orchestra.run.smithi032.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T09:14:44.973 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-13T09:14:44.973 INFO:teuthology.orchestra.run.smithi032.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T09:14:44.973 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T09:14:44.973 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T09:14:44.973 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T09:14:44.973 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T09:14:44.973 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T09:14:44.973 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T09:14:44.973 INFO:teuthology.orchestra.run.smithi032.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T09:14:44.974 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T09:14:44.974 INFO:teuthology.orchestra.run.smithi032.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T09:14:44.974 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-13T09:14:44.974 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T09:14:44.974 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T09:14:44.974 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T09:14:44.974 INFO:teuthology.orchestra.run.smithi032.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T09:14:44.974 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T09:14:44.974 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-13T09:14:44.974 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2024-09-13T09:14:44.975 INFO:teuthology.orchestra.run.smithi032.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-13T09:14:44.975 INFO:teuthology.orchestra.run.smithi032.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-13T09:14:44.975 INFO:teuthology.orchestra.run.smithi032.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-13T09:14:44.975 INFO:teuthology.orchestra.run.smithi032.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-13T09:14:44.975 INFO:teuthology.orchestra.run.smithi032.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-13T09:14:44.975 INFO:teuthology.orchestra.run.smithi032.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-13T09:14:44.975 INFO:teuthology.orchestra.run.smithi032.stdout: python3-watchdog 2024-09-13T09:14:44.975 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2024-09-13T09:14:44.975 INFO:teuthology.orchestra.run.smithi032.stdout: btrfs-tools 2024-09-13T09:14:45.045 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:14:45.045 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:14:45.045 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:14:45.045 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:14:45.045 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T09:14:45.046 INFO:teuthology.orchestra.run.smithi046.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T09:14:45.046 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:14:45.046 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2024-09-13T09:14:45.046 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T09:14:45.047 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T09:14:45.047 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-13T09:14:45.047 INFO:teuthology.orchestra.run.smithi046.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T09:14:45.048 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-13T09:14:45.048 INFO:teuthology.orchestra.run.smithi046.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T09:14:45.048 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T09:14:45.048 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T09:14:45.048 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T09:14:45.048 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T09:14:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T09:14:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T09:14:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T09:14:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T09:14:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T09:14:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-13T09:14:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T09:14:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T09:14:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T09:14:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T09:14:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T09:14:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-13T09:14:45.050 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2024-09-13T09:14:45.050 INFO:teuthology.orchestra.run.smithi046.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-13T09:14:45.050 INFO:teuthology.orchestra.run.smithi046.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-13T09:14:45.050 INFO:teuthology.orchestra.run.smithi046.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-13T09:14:45.050 INFO:teuthology.orchestra.run.smithi046.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-13T09:14:45.050 INFO:teuthology.orchestra.run.smithi046.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-13T09:14:45.050 INFO:teuthology.orchestra.run.smithi046.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-13T09:14:45.050 INFO:teuthology.orchestra.run.smithi046.stdout: python3-watchdog 2024-09-13T09:14:45.050 INFO:teuthology.orchestra.run.smithi046.stdout:Recommended packages: 2024-09-13T09:14:45.050 INFO:teuthology.orchestra.run.smithi046.stdout: btrfs-tools 2024-09-13T09:14:45.083 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2024-09-13T09:14:45.083 INFO:teuthology.orchestra.run.smithi032.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T09:14:45.083 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T09:14:45.083 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-13T09:14:45.084 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-13T09:14:45.084 INFO:teuthology.orchestra.run.smithi032.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-13T09:14:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-13T09:14:45.086 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T09:14:45.086 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T09:14:45.086 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T09:14:45.086 INFO:teuthology.orchestra.run.smithi032.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T09:14:45.086 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T09:14:45.086 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-13T09:14:45.086 INFO:teuthology.orchestra.run.smithi032.stdout: socat xmlstarlet 2024-09-13T09:14:45.145 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 109 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T09:14:45.145 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 163 MB of archives. 2024-09-13T09:14:45.145 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-13T09:14:45.145 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-13T09:14:45.158 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2024-09-13T09:14:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T09:14:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T09:14:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-13T09:14:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-13T09:14:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-13T09:14:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-13T09:14:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-13T09:14:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T09:14:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T09:14:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-13T09:14:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-13T09:14:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-13T09:14:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-13T09:14:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-13T09:14:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T09:14:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-13T09:14:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-13T09:14:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-13T09:14:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T09:14:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T09:14:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T09:14:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T09:14:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T09:14:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-13T09:14:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: socat xmlstarlet 2024-09-13T09:14:45.205 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T09:14:45.229 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-820-g48a7de7e-1jammy [3,451 kB] 2024-09-13T09:14:45.241 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T09:14:45.244 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T09:14:45.260 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T09:14:45.303 INFO:teuthology.orchestra.run.smithi046.stdout:0 upgraded, 109 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T09:14:45.304 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 163 MB of archives. 2024-09-13T09:14:45.304 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-13T09:14:45.304 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-820-g48a7de7e-1jammy [3,451 kB] 2024-09-13T09:14:45.314 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T09:14:45.319 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T09:14:45.331 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T09:14:45.334 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T09:14:45.335 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T09:14:45.335 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T09:14:45.344 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T09:14:45.344 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T09:14:45.345 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T09:14:45.346 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T09:14:45.349 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-13T09:14:45.354 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T09:14:45.354 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T09:14:45.354 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T09:14:45.371 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T09:14:45.372 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T09:14:45.372 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T09:14:45.377 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T09:14:45.377 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T09:14:45.377 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T09:14:45.378 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T09:14:45.388 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T09:14:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T09:14:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T09:14:45.406 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T09:14:45.407 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T09:14:45.408 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T09:14:45.409 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-820-g48a7de7e-1jammy [3,108 kB] 2024-09-13T09:14:45.413 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T09:14:45.413 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T09:14:45.414 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T09:14:45.415 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T09:14:45.423 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T09:14:45.425 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T09:14:45.426 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T09:14:45.441 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T09:14:45.442 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T09:14:45.445 INFO:teuthology.orchestra.run.smithi032.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T09:14:45.446 INFO:teuthology.orchestra.run.smithi032.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T09:14:45.447 INFO:teuthology.orchestra.run.smithi032.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T09:14:45.450 INFO:teuthology.orchestra.run.smithi032.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T09:14:45.461 INFO:teuthology.orchestra.run.smithi032.stdout:Get:47 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-820-g48a7de7e-1jammy [683 kB] 2024-09-13T09:14:45.474 INFO:teuthology.orchestra.run.smithi032.stdout:Get:48 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-820-g48a7de7e-1jammy [357 kB] 2024-09-13T09:14:45.480 INFO:teuthology.orchestra.run.smithi032.stdout:Get:49 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-820-g48a7de7e-1jammy [32.8 kB] 2024-09-13T09:14:45.481 INFO:teuthology.orchestra.run.smithi032.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-820-g48a7de7e-1jammy [180 kB] 2024-09-13T09:14:45.484 INFO:teuthology.orchestra.run.smithi032.stdout:Get:51 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-820-g48a7de7e-1jammy [63.8 kB] 2024-09-13T09:14:45.485 INFO:teuthology.orchestra.run.smithi032.stdout:Get:52 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-820-g48a7de7e-1jammy [334 kB] 2024-09-13T09:14:45.491 INFO:teuthology.orchestra.run.smithi032.stdout:Get:53 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-820-g48a7de7e-1jammy [5,934 kB] 2024-09-13T09:14:45.506 INFO:teuthology.orchestra.run.smithi032.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T09:14:45.507 INFO:teuthology.orchestra.run.smithi032.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T09:14:45.508 INFO:teuthology.orchestra.run.smithi032.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T09:14:45.508 INFO:teuthology.orchestra.run.smithi032.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T09:14:45.509 INFO:teuthology.orchestra.run.smithi032.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T09:14:45.509 INFO:teuthology.orchestra.run.smithi032.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T09:14:45.510 INFO:teuthology.orchestra.run.smithi032.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T09:14:45.512 INFO:teuthology.orchestra.run.smithi032.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T09:14:45.512 INFO:teuthology.orchestra.run.smithi032.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T09:14:45.525 INFO:teuthology.orchestra.run.smithi032.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T09:14:45.545 INFO:teuthology.orchestra.run.smithi032.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T09:14:45.547 INFO:teuthology.orchestra.run.smithi032.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T09:14:45.547 INFO:teuthology.orchestra.run.smithi032.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T09:14:45.562 INFO:teuthology.orchestra.run.smithi032.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T09:14:45.563 INFO:teuthology.orchestra.run.smithi032.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T09:14:45.563 INFO:teuthology.orchestra.run.smithi032.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T09:14:45.563 INFO:teuthology.orchestra.run.smithi032.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T09:14:45.564 INFO:teuthology.orchestra.run.smithi032.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T09:14:45.565 INFO:teuthology.orchestra.run.smithi032.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T09:14:45.565 INFO:teuthology.orchestra.run.smithi032.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T09:14:45.581 INFO:teuthology.orchestra.run.smithi032.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T09:14:45.585 INFO:teuthology.orchestra.run.smithi032.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T09:14:45.586 INFO:teuthology.orchestra.run.smithi032.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T09:14:45.588 INFO:teuthology.orchestra.run.smithi032.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T09:14:45.599 INFO:teuthology.orchestra.run.smithi032.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T09:14:45.602 INFO:teuthology.orchestra.run.smithi032.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T09:14:45.605 INFO:teuthology.orchestra.run.smithi032.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T09:14:45.607 INFO:teuthology.orchestra.run.smithi032.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-820-g48a7de7e-1jammy [112 kB] 2024-09-13T09:14:45.609 INFO:teuthology.orchestra.run.smithi032.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-820-g48a7de7e-1jammy [445 kB] 2024-09-13T09:14:45.609 INFO:teuthology.orchestra.run.smithi032.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T09:14:45.612 INFO:teuthology.orchestra.run.smithi032.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T09:14:45.612 INFO:teuthology.orchestra.run.smithi032.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T09:14:45.615 INFO:teuthology.orchestra.run.smithi032.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-820-g48a7de7e-1jammy [22.0 MB] 2024-09-13T09:14:45.616 INFO:teuthology.orchestra.run.smithi032.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T09:14:45.624 INFO:teuthology.orchestra.run.smithi032.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T09:14:45.624 INFO:teuthology.orchestra.run.smithi032.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T09:14:45.624 INFO:teuthology.orchestra.run.smithi032.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T09:14:45.631 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T09:14:45.756 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-820-g48a7de7e-1jammy [3,108 kB] 2024-09-13T09:14:45.804 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T09:14:45.817 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T09:14:45.891 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T09:14:46.085 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T09:14:46.094 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T09:14:46.095 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-820-g48a7de7e-1jammy [683 kB] 2024-09-13T09:14:46.113 INFO:teuthology.orchestra.run.smithi032.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-820-g48a7de7e-1jammy [5,005 kB] 2024-09-13T09:14:46.117 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-820-g48a7de7e-1jammy [357 kB] 2024-09-13T09:14:46.122 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T09:14:46.131 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T09:14:46.132 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T09:14:46.132 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-820-g48a7de7e-1jammy [32.8 kB] 2024-09-13T09:14:46.133 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T09:14:46.133 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-820-g48a7de7e-1jammy [180 kB] 2024-09-13T09:14:46.138 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-820-g48a7de7e-1jammy [63.8 kB] 2024-09-13T09:14:46.140 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-820-g48a7de7e-1jammy [334 kB] 2024-09-13T09:14:46.152 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T09:14:46.152 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-820-g48a7de7e-1jammy [5,934 kB] 2024-09-13T09:14:46.153 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T09:14:46.154 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T09:14:46.168 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T09:14:46.174 INFO:teuthology.orchestra.run.smithi032.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-820-g48a7de7e-1jammy [241 kB] 2024-09-13T09:14:46.177 INFO:teuthology.orchestra.run.smithi032.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-820-g48a7de7e-1jammy [130 kB] 2024-09-13T09:14:46.178 INFO:teuthology.orchestra.run.smithi032.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-820-g48a7de7e-1jammy [1,494 kB] 2024-09-13T09:14:46.196 INFO:teuthology.orchestra.run.smithi032.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-820-g48a7de7e-1jammy [6,085 kB] 2024-09-13T09:14:46.239 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T09:14:46.239 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T09:14:46.240 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T09:14:46.241 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T09:14:46.242 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T09:14:46.242 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T09:14:46.249 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T09:14:46.249 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T09:14:46.249 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T09:14:46.321 INFO:teuthology.orchestra.run.smithi032.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-820-g48a7de7e-1jammy [22.4 MB] 2024-09-13T09:14:46.331 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T09:14:46.333 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T09:14:46.333 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T09:14:46.334 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T09:14:46.336 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T09:14:46.338 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T09:14:46.339 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T09:14:46.350 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T09:14:46.413 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T09:14:46.413 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T09:14:46.432 INFO:teuthology.orchestra.run.smithi046.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T09:14:46.434 INFO:teuthology.orchestra.run.smithi046.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T09:14:46.438 INFO:teuthology.orchestra.run.smithi046.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T09:14:46.440 INFO:teuthology.orchestra.run.smithi046.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T09:14:46.495 INFO:teuthology.orchestra.run.smithi046.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T09:14:46.496 INFO:teuthology.orchestra.run.smithi046.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T09:14:46.500 INFO:teuthology.orchestra.run.smithi046.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T09:14:46.502 INFO:teuthology.orchestra.run.smithi046.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T09:14:46.576 INFO:teuthology.orchestra.run.smithi046.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T09:14:46.580 INFO:teuthology.orchestra.run.smithi046.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T09:14:46.601 INFO:teuthology.orchestra.run.smithi046.stdout:Get:54 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-820-g48a7de7e-1jammy [112 kB] 2024-09-13T09:14:46.608 INFO:teuthology.orchestra.run.smithi046.stdout:Get:55 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-820-g48a7de7e-1jammy [445 kB] 2024-09-13T09:14:46.643 INFO:teuthology.orchestra.run.smithi046.stdout:Get:56 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-820-g48a7de7e-1jammy [22.0 MB] 2024-09-13T09:14:46.742 INFO:teuthology.orchestra.run.smithi046.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T09:14:46.744 INFO:teuthology.orchestra.run.smithi046.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T09:14:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T09:14:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T09:14:46.749 INFO:teuthology.orchestra.run.smithi046.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T09:14:46.750 INFO:teuthology.orchestra.run.smithi046.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T09:14:46.751 INFO:teuthology.orchestra.run.smithi046.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T09:14:46.756 INFO:teuthology.orchestra.run.smithi046.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T09:14:46.756 INFO:teuthology.orchestra.run.smithi046.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T09:14:46.827 INFO:teuthology.orchestra.run.smithi046.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T09:14:46.872 INFO:teuthology.orchestra.run.smithi046.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T09:14:46.876 INFO:teuthology.orchestra.run.smithi046.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T09:14:46.877 INFO:teuthology.orchestra.run.smithi046.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T09:14:46.896 INFO:teuthology.orchestra.run.smithi032.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-820-g48a7de7e-1jammy [14.2 kB] 2024-09-13T09:14:46.897 INFO:teuthology.orchestra.run.smithi032.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-820-g48a7de7e-1jammy [834 kB] 2024-09-13T09:14:46.912 INFO:teuthology.orchestra.run.smithi032.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-820-g48a7de7e-1jammy [2,159 kB] 2024-09-13T09:14:46.929 INFO:teuthology.orchestra.run.smithi046.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T09:14:46.929 INFO:teuthology.orchestra.run.smithi046.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T09:14:46.931 INFO:teuthology.orchestra.run.smithi046.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T09:14:46.931 INFO:teuthology.orchestra.run.smithi046.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T09:14:46.932 INFO:teuthology.orchestra.run.smithi046.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T09:14:46.933 INFO:teuthology.orchestra.run.smithi046.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T09:14:46.949 INFO:teuthology.orchestra.run.smithi032.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-820-g48a7de7e-1jammy [240 kB] 2024-09-13T09:14:46.953 INFO:teuthology.orchestra.run.smithi032.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-820-g48a7de7e-1jammy [145 kB] 2024-09-13T09:14:46.955 INFO:teuthology.orchestra.run.smithi032.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-820-g48a7de7e-1jammy [2,233 kB] 2024-09-13T09:14:46.979 INFO:teuthology.orchestra.run.smithi032.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-820-g48a7de7e-1jammy [8,625 kB] 2024-09-13T09:14:46.990 INFO:teuthology.orchestra.run.smithi046.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T09:14:46.991 INFO:teuthology.orchestra.run.smithi046.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T09:14:46.997 INFO:teuthology.orchestra.run.smithi046.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T09:14:47.071 INFO:teuthology.orchestra.run.smithi046.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T09:14:47.075 INFO:teuthology.orchestra.run.smithi046.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T09:14:47.076 INFO:teuthology.orchestra.run.smithi046.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T09:14:47.084 INFO:teuthology.orchestra.run.smithi046.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T09:14:47.091 INFO:teuthology.orchestra.run.smithi046.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T09:14:47.102 INFO:teuthology.orchestra.run.smithi046.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T09:14:47.109 INFO:teuthology.orchestra.run.smithi046.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T09:14:47.153 INFO:teuthology.orchestra.run.smithi046.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T09:14:47.156 INFO:teuthology.orchestra.run.smithi046.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T09:14:47.173 INFO:teuthology.orchestra.run.smithi046.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T09:14:47.192 INFO:teuthology.orchestra.run.smithi032.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-820-g48a7de7e-1jammy [14.3 kB] 2024-09-13T09:14:47.192 INFO:teuthology.orchestra.run.smithi032.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-820-g48a7de7e-1jammy [42.8 MB] 2024-09-13T09:14:47.234 INFO:teuthology.orchestra.run.smithi046.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T09:14:47.235 INFO:teuthology.orchestra.run.smithi046.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T09:14:47.709 INFO:teuthology.orchestra.run.smithi046.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-820-g48a7de7e-1jammy [5,005 kB] 2024-09-13T09:14:47.828 INFO:teuthology.orchestra.run.smithi046.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-820-g48a7de7e-1jammy [241 kB] 2024-09-13T09:14:47.833 INFO:teuthology.orchestra.run.smithi046.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-820-g48a7de7e-1jammy [130 kB] 2024-09-13T09:14:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-820-g48a7de7e-1jammy [1,494 kB] 2024-09-13T09:14:47.903 INFO:teuthology.orchestra.run.smithi046.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-820-g48a7de7e-1jammy [6,085 kB] 2024-09-13T09:14:48.136 INFO:teuthology.orchestra.run.smithi046.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-820-g48a7de7e-1jammy [22.4 MB] 2024-09-13T09:14:48.543 INFO:teuthology.orchestra.run.smithi032.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-820-g48a7de7e-1jammy [122 kB] 2024-09-13T09:14:48.546 INFO:teuthology.orchestra.run.smithi032.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-820-g48a7de7e-1jammy [38.3 kB] 2024-09-13T09:14:48.547 INFO:teuthology.orchestra.run.smithi032.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-820-g48a7de7e-1jammy [11.7 MB] 2024-09-13T09:14:48.814 INFO:teuthology.orchestra.run.smithi032.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-820-g48a7de7e-1jammy [93.1 kB] 2024-09-13T09:14:48.934 INFO:teuthology.orchestra.run.smithi046.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-820-g48a7de7e-1jammy [14.2 kB] 2024-09-13T09:14:48.935 INFO:teuthology.orchestra.run.smithi046.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-820-g48a7de7e-1jammy [834 kB] 2024-09-13T09:14:48.951 INFO:teuthology.orchestra.run.smithi046.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-820-g48a7de7e-1jammy [2,159 kB] 2024-09-13T09:14:48.991 INFO:teuthology.orchestra.run.smithi046.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-820-g48a7de7e-1jammy [240 kB] 2024-09-13T09:14:48.994 INFO:teuthology.orchestra.run.smithi046.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-820-g48a7de7e-1jammy [145 kB] 2024-09-13T09:14:48.997 INFO:teuthology.orchestra.run.smithi046.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-820-g48a7de7e-1jammy [2,233 kB] 2024-09-13T09:14:49.022 INFO:teuthology.orchestra.run.smithi046.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-820-g48a7de7e-1jammy [8,625 kB] 2024-09-13T09:14:49.139 INFO:teuthology.orchestra.run.smithi046.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-820-g48a7de7e-1jammy [14.3 kB] 2024-09-13T09:14:49.139 INFO:teuthology.orchestra.run.smithi046.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-820-g48a7de7e-1jammy [42.8 MB] 2024-09-13T09:14:49.408 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 163 MB in 4s (43.7 MB/s) 2024-09-13T09:14:49.523 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-13T09:14:49.612 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T09:14:49.618 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-13T09:14:49.688 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T09:14:49.950 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T09:14:49.976 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T09:14:50.022 INFO:teuthology.orchestra.run.smithi046.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-820-g48a7de7e-1jammy [122 kB] 2024-09-13T09:14:50.027 INFO:teuthology.orchestra.run.smithi046.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-820-g48a7de7e-1jammy [38.3 kB] 2024-09-13T09:14:50.027 INFO:teuthology.orchestra.run.smithi046.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-820-g48a7de7e-1jammy [11.7 MB] 2024-09-13T09:14:50.033 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T09:14:50.198 INFO:teuthology.orchestra.run.smithi046.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-820-g48a7de7e-1jammy [93.1 kB] 2024-09-13T09:14:50.353 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T09:14:50.380 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T09:14:50.419 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T09:14:50.752 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 163 MB in 5s (32.3 MB/s) 2024-09-13T09:14:50.764 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T09:14:50.793 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T09:14:50.830 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T09:14:50.891 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-13T09:14:50.989 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T09:14:50.995 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-13T09:14:51.081 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T09:14:51.217 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T09:14:51.244 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T09:14:51.284 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:14:51.401 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T09:14:51.428 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T09:14:51.468 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T09:14:51.662 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T09:14:51.688 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T09:14:51.728 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:14:51.796 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T09:14:51.823 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T09:14:51.863 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T09:14:52.065 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T09:14:52.092 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T09:14:52.131 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:14:52.199 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T09:14:52.226 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T09:14:52.266 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T09:14:52.501 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T09:14:52.528 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T09:14:52.568 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T09:14:52.644 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T09:14:52.672 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T09:14:52.711 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:14:52.904 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librados2. 2024-09-13T09:14:52.930 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../008-librados2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:52.970 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librados2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T09:14:53.107 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T09:14:53.139 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:14:53.381 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librbd1. 2024-09-13T09:14:53.397 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../009-librbd1_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:53.431 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librbd1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:53.475 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T09:14:53.502 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T09:14:53.542 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:14:53.895 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T09:14:53.900 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T09:14:53.922 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T09:14:53.927 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:53.962 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T09:14:53.967 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:54.248 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librados2. 2024-09-13T09:14:54.253 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rados. 2024-09-13T09:14:54.263 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../008-librados2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:54.268 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../011-python3-rados_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:54.297 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librados2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:54.311 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rados (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:54.639 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T09:14:54.665 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:14:54.705 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-argparse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:54.742 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librbd1. 2024-09-13T09:14:54.769 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../009-librbd1_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:54.809 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librbd1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:54.916 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T09:14:54.931 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:54.966 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cephfs (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:55.235 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T09:14:55.245 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T09:14:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:14:55.271 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:55.285 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:55.311 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:55.563 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T09:14:55.590 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T09:14:55.597 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rados. 2024-09-13T09:14:55.624 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../011-python3-rados_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:55.629 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T09:14:55.664 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rados (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:55.958 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T09:14:55.984 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T09:14:55.991 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T09:14:56.018 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:14:56.024 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T09:14:56.067 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-argparse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:56.285 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T09:14:56.311 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T09:14:56.311 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:56.327 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:56.351 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rbd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:56.361 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cephfs (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:56.671 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T09:14:56.672 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T09:14:56.698 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T09:14:56.699 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:14:56.729 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T09:14:56.738 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:57.032 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librgw2. 2024-09-13T09:14:57.058 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../019-librgw2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:57.099 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librgw2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:57.108 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T09:14:57.135 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T09:14:57.175 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T09:14:57.494 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T09:14:57.521 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T09:14:57.561 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T09:14:57.797 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T09:14:57.814 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T09:14:57.824 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:57.841 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:57.855 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rbd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:57.967 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:58.191 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T09:14:58.218 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T09:14:58.258 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T09:14:58.329 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T09:14:58.355 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T09:14:58.395 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T09:14:58.586 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librgw2. 2024-09-13T09:14:58.613 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../019-librgw2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:58.652 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librgw2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:58.664 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T09:14:58.691 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:58.723 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libradosstriper1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:59.151 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-common. 2024-09-13T09:14:59.155 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T09:14:59.171 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:59.178 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../023-ceph-common_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:59.205 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:59.217 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:14:59.491 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T09:14:59.507 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T09:14:59.549 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T09:14:59.877 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T09:14:59.904 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:14:59.944 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libradosstriper1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:00.372 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-common. 2024-09-13T09:15:00.399 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../023-ceph-common_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:00.438 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:00.515 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-base. 2024-09-13T09:15:00.542 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../024-ceph-base_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:00.591 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-base (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:01.186 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T09:15:01.212 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T09:15:01.252 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T09:15:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T09:15:01.529 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T09:15:01.563 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T09:15:01.686 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-base. 2024-09-13T09:15:01.713 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../024-ceph-base_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:01.762 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-base (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:01.858 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T09:15:01.874 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T09:15:01.908 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T09:15:02.210 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T09:15:02.237 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T09:15:02.277 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T09:15:02.549 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T09:15:02.576 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T09:15:02.604 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T09:15:02.607 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T09:15:02.632 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T09:15:02.671 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T09:15:02.927 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T09:15:02.954 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T09:15:02.993 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T09:15:03.158 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T09:15:03.185 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T09:15:03.216 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T09:15:03.321 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T09:15:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T09:15:03.388 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T09:15:03.510 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-portend. 2024-09-13T09:15:03.537 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-13T09:15:03.577 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T09:15:03.691 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T09:15:03.718 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T09:15:03.757 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T09:15:03.880 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T09:15:03.907 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T09:15:03.946 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T09:15:04.019 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T09:15:04.045 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T09:15:04.085 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T09:15:04.216 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T09:15:04.244 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T09:15:04.274 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T09:15:04.388 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T09:15:04.415 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T09:15:04.455 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T09:15:04.611 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T09:15:04.638 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T09:15:04.677 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T09:15:04.757 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-portend. 2024-09-13T09:15:04.784 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-13T09:15:04.824 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T09:15:04.989 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T09:15:05.015 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T09:15:05.055 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T09:15:05.085 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T09:15:05.112 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T09:15:05.144 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T09:15:05.316 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T09:15:05.343 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T09:15:05.374 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T09:15:05.438 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T09:15:05.465 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T09:15:05.506 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T09:15:05.669 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-mako. 2024-09-13T09:15:05.696 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T09:15:05.735 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T09:15:05.850 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T09:15:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T09:15:05.916 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T09:15:06.021 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T09:15:06.048 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T09:15:06.088 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T09:15:06.220 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T09:15:06.247 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T09:15:06.286 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T09:15:06.399 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T09:15:06.415 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T09:15:06.457 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T09:15:06.572 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T09:15:06.599 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T09:15:06.639 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T09:15:06.752 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webob. 2024-09-13T09:15:06.779 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T09:15:06.893 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T09:15:06.900 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-mako. 2024-09-13T09:15:06.927 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T09:15:06.966 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T09:15:07.255 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T09:15:07.281 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T09:15:07.320 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T09:15:07.324 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T09:15:07.347 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T09:15:07.387 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T09:15:07.607 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T09:15:07.634 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T09:15:07.648 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T09:15:07.674 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T09:15:07.675 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T09:15:07.706 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T09:15:07.944 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-paste. 2024-09-13T09:15:07.970 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T09:15:07.983 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webob. 2024-09-13T09:15:08.010 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T09:15:08.010 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T09:15:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T09:15:08.403 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T09:15:08.430 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T09:15:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T09:15:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T09:15:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T09:15:08.529 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T09:15:08.777 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T09:15:08.798 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T09:15:08.800 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T09:15:08.824 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T09:15:08.840 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T09:15:08.864 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T09:15:09.125 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-paste. 2024-09-13T09:15:09.142 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T09:15:09.143 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T09:15:09.170 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T09:15:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T09:15:09.210 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T09:15:09.495 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T09:15:09.513 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T09:15:09.521 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T09:15:09.548 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T09:15:09.554 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T09:15:09.587 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T09:15:09.798 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T09:15:09.825 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T09:15:09.856 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T09:15:09.967 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T09:15:09.994 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T09:15:10.033 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T09:15:10.118 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T09:15:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T09:15:10.175 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T09:15:10.328 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-idna. 2024-09-13T09:15:10.355 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T09:15:10.394 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T09:15:10.463 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T09:15:10.489 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T09:15:10.529 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T09:15:10.647 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T09:15:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T09:15:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T09:15:10.892 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T09:15:10.919 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T09:15:10.958 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T09:15:11.017 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests. 2024-09-13T09:15:11.044 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T09:15:11.083 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T09:15:11.312 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-idna. 2024-09-13T09:15:11.339 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T09:15:11.378 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T09:15:11.428 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T09:15:11.455 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T09:15:11.497 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T09:15:11.673 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T09:15:11.700 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T09:15:11.739 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T09:15:11.806 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T09:15:11.833 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:15:11.872 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:12.042 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests. 2024-09-13T09:15:12.058 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T09:15:12.092 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T09:15:12.235 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T09:15:12.251 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:12.285 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:12.529 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T09:15:12.556 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T09:15:12.580 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T09:15:12.595 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T09:15:12.606 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T09:15:12.646 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T09:15:12.865 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T09:15:12.892 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:15:12.923 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:12.949 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T09:15:12.976 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T09:15:13.015 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T09:15:13.328 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T09:15:13.355 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:13.368 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T09:15:13.394 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:13.395 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:13.435 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:13.689 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T09:15:13.705 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T09:15:13.738 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T09:15:13.788 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T09:15:13.804 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:13.837 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mon (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:13.992 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T09:15:14.019 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T09:15:14.058 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T09:15:14.411 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T09:15:14.439 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:14.449 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T09:15:14.476 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:14.478 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:14.516 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-osd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:14.881 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T09:15:14.908 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:14.947 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mon (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:15.509 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T09:15:15.536 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:15.567 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-osd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:15.602 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph. 2024-09-13T09:15:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../060-ceph_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:15.661 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:15.947 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T09:15:15.974 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:16.013 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:16.350 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T09:15:16.377 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:16.425 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mds (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:16.630 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph. 2024-09-13T09:15:16.656 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../060-ceph_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:16.696 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:16.811 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephadm. 2024-09-13T09:15:16.838 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../063-cephadm_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:16.877 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:17.015 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T09:15:17.042 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:17.082 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:17.197 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T09:15:17.224 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T09:15:17.263 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T09:15:17.402 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T09:15:17.417 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:17.583 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T09:15:17.610 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T09:15:17.649 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T09:15:17.893 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mds (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:18.649 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephadm. 2024-09-13T09:15:18.676 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../063-cephadm_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:18.715 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:19.010 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T09:15:19.037 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T09:15:19.076 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T09:15:19.404 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T09:15:19.420 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T09:15:19.462 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T09:15:19.829 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-babel. 2024-09-13T09:15:19.846 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T09:15:19.896 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T09:15:20.183 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T09:15:20.209 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T09:15:20.240 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T09:15:20.526 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-babel. 2024-09-13T09:15:20.544 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T09:15:20.553 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T09:15:20.571 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:15:20.592 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T09:15:20.610 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:20.920 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T09:15:20.939 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T09:15:20.947 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T09:15:20.966 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T09:15:20.986 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T09:15:21.005 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T09:15:21.275 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-py. 2024-09-13T09:15:21.302 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-13T09:15:21.306 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T09:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:15:21.341 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T09:15:21.365 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:21.669 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T09:15:21.697 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T09:15:21.701 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T09:15:21.729 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T09:15:21.736 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T09:15:21.768 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T09:15:21.980 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-toml. 2024-09-13T09:15:22.008 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-13T09:15:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-py. 2024-09-13T09:15:22.039 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T09:15:22.057 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-13T09:15:22.087 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T09:15:22.341 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T09:15:22.369 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T09:15:22.399 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T09:15:22.408 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T09:15:22.427 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T09:15:22.465 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T09:15:22.711 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T09:15:22.727 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T09:15:22.761 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T09:15:22.768 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-toml. 2024-09-13T09:15:22.796 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-13T09:15:23.010 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T09:15:23.048 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-routes. 2024-09-13T09:15:23.075 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T09:15:23.114 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T09:15:23.329 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T09:15:23.357 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T09:15:23.396 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T09:15:23.436 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T09:15:23.464 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:15:23.502 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:23.766 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T09:15:23.794 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T09:15:23.833 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T09:15:24.135 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-routes. 2024-09-13T09:15:24.163 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T09:15:24.202 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T09:15:24.555 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T09:15:24.583 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:15:24.622 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:24.774 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T09:15:24.791 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T09:15:24.832 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T09:15:25.345 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T09:15:25.372 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T09:15:25.411 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T09:15:25.756 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T09:15:25.784 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T09:15:25.793 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T09:15:25.810 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T09:15:25.814 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T09:15:25.843 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T09:15:26.084 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T09:15:26.111 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T09:15:26.150 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T09:15:26.330 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T09:15:26.358 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T09:15:26.397 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T09:15:26.759 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T09:15:26.786 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T09:15:26.825 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T09:15:26.859 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T09:15:26.887 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:15:26.925 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:27.103 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T09:15:27.130 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T09:15:27.161 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T09:15:27.870 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T09:15:27.894 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T09:15:27.897 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T09:15:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:15:27.936 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T09:15:27.960 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:28.316 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T09:15:28.343 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T09:15:28.382 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T09:15:28.719 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T09:15:28.747 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T09:15:28.820 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T09:15:28.888 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T09:15:28.916 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T09:15:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T09:15:29.130 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T09:15:29.158 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-13T09:15:29.197 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T09:15:29.359 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T09:15:29.387 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T09:15:29.425 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T09:15:29.737 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T09:15:29.742 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T09:15:29.765 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T09:15:29.770 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T09:15:29.804 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T09:15:29.808 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T09:15:30.095 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T09:15:30.112 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T09:15:30.123 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T09:15:30.145 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T09:15:30.151 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-13T09:15:30.190 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T09:15:30.423 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T09:15:30.451 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T09:15:30.489 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T09:15:30.510 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T09:15:30.538 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T09:15:30.576 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T09:15:30.818 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T09:15:30.845 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T09:15:30.880 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T09:15:30.884 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T09:15:30.907 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T09:15:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T09:15:31.199 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T09:15:31.227 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T09:15:31.257 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T09:15:31.573 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T09:15:31.590 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:15:31.602 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T09:15:31.630 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T09:15:31.631 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:31.668 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T09:15:31.917 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T09:15:31.945 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T09:15:31.975 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T09:15:32.270 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T09:15:32.298 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T09:15:32.337 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T09:15:32.357 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T09:15:32.385 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:15:32.424 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:32.606 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package jq. 2024-09-13T09:15:32.634 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T09:15:32.672 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T09:15:32.710 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T09:15:32.726 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T09:15:32.759 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T09:15:32.917 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package socat. 2024-09-13T09:15:32.933 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T09:15:32.975 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T09:15:33.082 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T09:15:33.110 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T09:15:33.278 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T09:15:33.279 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T09:15:33.295 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T09:15:33.328 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T09:15:33.523 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package jq. 2024-09-13T09:15:33.540 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T09:15:33.581 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T09:15:33.691 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-test. 2024-09-13T09:15:33.708 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../096-ceph-test_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:33.740 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-test (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:33.834 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package socat. 2024-09-13T09:15:33.862 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T09:15:33.900 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T09:15:34.204 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T09:15:34.220 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T09:15:34.262 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T09:15:34.633 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-test. 2024-09-13T09:15:34.649 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../096-ceph-test_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:34.691 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-test (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:35.661 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T09:15:35.689 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:15:35.736 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-volume (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:36.040 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T09:15:36.068 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:36.107 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs-dev (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:36.477 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T09:15:36.504 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T09:15:36.543 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T09:15:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T09:15:36.681 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:15:36.719 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-volume (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:36.906 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T09:15:36.923 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T09:15:36.973 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T09:15:37.023 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T09:15:37.040 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:37.082 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs-dev (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:37.393 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T09:15:37.410 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T09:15:37.421 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T09:15:37.437 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T09:15:37.459 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T09:15:37.476 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T09:15:37.815 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T09:15:37.842 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T09:15:37.855 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T09:15:37.883 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T09:15:37.889 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T09:15:37.929 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T09:15:38.291 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T09:15:38.319 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T09:15:38.365 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T09:15:38.367 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T09:15:38.395 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T09:15:38.442 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T09:15:38.829 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T09:15:38.857 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T09:15:38.866 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T09:15:38.895 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T09:15:38.903 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T09:15:38.952 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T09:15:39.230 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T09:15:39.247 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T09:15:39.280 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T09:15:39.382 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T09:15:39.410 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T09:15:39.456 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T09:15:39.549 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T09:15:39.566 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T09:15:39.599 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T09:15:39.960 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T09:15:39.989 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T09:15:40.026 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T09:15:40.321 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T09:15:40.328 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package radosgw. 2024-09-13T09:15:40.349 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T09:15:40.356 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../107-radosgw_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:40.379 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T09:15:40.394 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking radosgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:40.623 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T09:15:40.640 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T09:15:40.673 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T09:15:41.123 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T09:15:41.139 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:41.181 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking rbd-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:41.352 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package radosgw. 2024-09-13T09:15:41.368 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../107-radosgw_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:41.402 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking radosgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:41.485 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T09:15:41.743 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T09:15:41.875 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T09:15:42.054 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T09:15:42.071 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:15:42.113 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking rbd-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:42.123 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T09:15:42.426 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T09:15:42.474 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T09:15:42.684 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T09:15:42.794 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T09:15:42.898 INFO:teuthology.orchestra.run.smithi032.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T09:15:42.981 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:43.061 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T09:15:43.415 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T09:15:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T09:15:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:44.046 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user cephadm....done 2024-09-13T09:15:44.283 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T09:15:44.461 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user cephadm....done 2024-09-13T09:15:44.618 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T09:15:44.621 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T09:15:44.919 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T09:15:44.941 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T09:15:45.042 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T09:15:45.204 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T09:15:45.285 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T09:15:45.330 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T09:15:45.545 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T09:15:45.589 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T09:15:45.788 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T09:15:45.857 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T09:15:45.975 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T09:15:46.106 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T09:15:46.301 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T09:15:46.474 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T09:15:46.670 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T09:15:46.717 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T09:15:46.946 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T09:15:47.010 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T09:15:47.222 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T09:15:47.253 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T09:15:47.490 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T09:15:47.504 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T09:15:47.738 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-argparse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:47.766 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T09:15:48.014 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T09:15:48.025 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-argparse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:48.140 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T09:15:48.285 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T09:15:48.400 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T09:15:48.501 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T09:15:48.515 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T09:15:48.852 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T09:15:48.978 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T09:15:48.995 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T09:15:49.121 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T09:15:49.237 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T09:15:49.497 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T09:15:49.505 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T09:15:49.631 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T09:15:49.806 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T09:15:49.899 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T09:15:50.158 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T09:15:50.451 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T09:15:50.500 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T09:15:50.793 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T09:15:50.894 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T09:15:51.003 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T09:15:51.145 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T09:15:51.296 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T09:15:51.404 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T09:15:51.555 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T09:15:51.672 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T09:15:51.681 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T09:15:51.931 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T09:15:51.982 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T09:15:52.341 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T09:15:52.358 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T09:15:52.601 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T09:15:52.634 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T09:15:52.985 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T09:15:52.986 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T09:15:53.245 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T09:15:53.287 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T09:15:53.371 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:53.655 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T09:15:53.689 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T09:15:53.923 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T09:15:54.015 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T09:15:54.049 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:15:54.367 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T09:15:54.685 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T09:15:55.304 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:15:55.422 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T09:15:55.681 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T09:15:55.824 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T09:15:55.950 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T09:15:55.965 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:15:56.099 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T09:15:56.268 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T09:15:56.375 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T09:15:56.510 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T09:15:56.527 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T09:15:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T09:15:56.778 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:15:56.879 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T09:15:56.988 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T09:15:57.070 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T09:15:57.315 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T09:15:57.330 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T09:15:57.574 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T09:15:57.597 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:15:57.724 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T09:15:57.833 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T09:15:57.841 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T09:15:58.118 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T09:15:58.134 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T09:15:58.377 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T09:15:58.402 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T09:15:58.653 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T09:15:58.686 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T09:15:58.756 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T09:15:58.945 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T09:15:59.031 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T09:15:59.157 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T09:15:59.230 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T09:15:59.458 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T09:15:59.506 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T09:15:59.632 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T09:15:59.759 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T09:15:59.933 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T09:16:00.002 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T09:16:00.059 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T09:16:00.335 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T09:16:00.383 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T09:16:00.654 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T09:16:00.678 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T09:16:00.772 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T09:16:00.929 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T09:16:01.182 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T09:16:01.344 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T09:16:01.450 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T09:16:01.623 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T09:16:01.701 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:16:01.741 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T09:16:01.801 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T09:16:02.103 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T09:16:02.159 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T09:16:02.396 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T09:16:02.410 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T09:16:02.661 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:16:02.672 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T09:16:02.770 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T09:16:02.798 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T09:16:03.066 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T09:16:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T09:16:03.350 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T09:16:03.406 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T09:16:03.628 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T09:16:03.690 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T09:16:03.808 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T09:16:04.019 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T09:16:04.226 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T09:16:04.312 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T09:16:04.501 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T09:16:04.690 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T09:16:04.811 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T09:16:04.815 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librados2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:04.941 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librgw2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:05.067 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:05.179 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T09:16:05.193 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:05.319 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libradosstriper1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:05.480 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T09:16:05.531 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T09:16:05.815 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librbd1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:05.849 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T09:16:05.942 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:05.975 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librados2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:06.068 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:06.101 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librgw2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:06.202 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:06.286 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T09:16:06.286 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T09:16:06.302 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:06.404 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libradosstriper1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:06.625 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T09:16:06.745 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs-dev (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:06.902 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librbd1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:06.904 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rados (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:07.028 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:07.047 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T09:16:07.154 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:07.375 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T09:16:07.375 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T09:16:07.889 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs-dev (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:08.015 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rados (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:08.140 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T09:16:08.326 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rbd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up rbd-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:08.561 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:08.663 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cephfs (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:08.773 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:09.332 INFO:teuthology.orchestra.run.smithi032.stdout:Adding group ceph....done 2024-09-13T09:16:09.437 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rbd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:09.562 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up rbd-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:09.688 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:09.800 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user ceph....done 2024-09-13T09:16:09.814 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cephfs (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:09.940 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:09.968 INFO:teuthology.orchestra.run.smithi032.stdout:Setting system user ceph properties....done 2024-09-13T09:16:09.987 INFO:teuthology.orchestra.run.smithi032.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T09:16:10.132 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T09:16:10.467 INFO:teuthology.orchestra.run.smithi046.stdout:Adding group ceph....done 2024-09-13T09:16:10.513 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T09:16:10.801 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user ceph....done 2024-09-13T09:16:10.970 INFO:teuthology.orchestra.run.smithi046.stdout:Setting system user ceph properties....done 2024-09-13T09:16:10.973 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-test (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:10.990 INFO:teuthology.orchestra.run.smithi046.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T09:16:11.108 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up radosgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:11.133 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T09:16:11.445 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T09:16:11.577 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:16:11.578 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:16:11.908 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-test (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:12.034 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up radosgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:12.043 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-base (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:12.460 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T09:16:12.482 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:16:12.482 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:16:12.919 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-base (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:12.937 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mds (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:13.154 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T09:16:13.155 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T09:16:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T09:16:13.607 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:13.704 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mds (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:13.833 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T09:16:13.833 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T09:16:13.945 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T09:16:13.945 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T09:16:14.300 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-osd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:14.414 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:14.579 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T09:16:14.579 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T09:16:14.646 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T09:16:14.647 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T09:16:15.069 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:15.115 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-osd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:15.228 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:15.404 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mon (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:15.413 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T09:16:15.413 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T09:16:15.644 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T09:16:15.644 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T09:16:15.867 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:15.984 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:16.102 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mon (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:16.169 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:16.311 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T09:16:16.311 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T09:16:16.353 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:16.504 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:16.705 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-volume (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:16.770 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:16.946 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:17.070 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:17.100 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T09:16:17.306 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-volume (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:16:17.659 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T09:16:18.018 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T09:16:18.517 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T09:16:18.524 DEBUG:teuthology.parallel:result is None 2024-09-13T09:16:18.843 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T09:16:19.326 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T09:16:19.331 DEBUG:teuthology.parallel:result is None 2024-09-13T09:16:19.332 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:16:19.472 DEBUG:teuthology.orchestra.run.smithi032:> dpkg-query -W -f '${Version}' ceph 2024-09-13T09:16:19.492 INFO:teuthology.orchestra.run.smithi032.stdout:18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:16:19.493 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:16:19.493 INFO:teuthology.task.install:The correct ceph version 18.2.4-820-g48a7de7e-1jammy is installed. 2024-09-13T09:16:19.495 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:16:19.631 DEBUG:teuthology.orchestra.run.smithi046:> dpkg-query -W -f '${Version}' ceph 2024-09-13T09:16:19.651 INFO:teuthology.orchestra.run.smithi046.stdout:18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:16:19.651 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:16:19.651 INFO:teuthology.task.install:The correct ceph version 18.2.4-820-g48a7de7e-1jammy is installed. 2024-09-13T09:16:19.653 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T09:16:19.653 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:19.653 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T09:16:19.668 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:19.668 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T09:16:19.707 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T09:16:19.708 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:19.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T09:16:19.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T09:16:19.779 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:19.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T09:16:19.793 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T09:16:19.850 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T09:16:19.850 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:19.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T09:16:19.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T09:16:19.919 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:19.919 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T09:16:19.933 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T09:16:19.985 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T09:16:19.986 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:19.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T09:16:20.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T09:16:20.055 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:20.056 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T09:16:20.070 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T09:16:20.125 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T09:16:20.209 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T09:16:20.210 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2024-09-13T09:16:20.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /var/log/ceph 2024-09-13T09:16:20.225 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T09:16:20.226 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T09:16:20.268 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T09:16:20.281 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T09:16:20.281 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T09:16:20.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T09:16:20.348 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T09:16:20.348 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache max dirty': 0}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T09:16:20.349 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022', 'branch': 'reef', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7901022', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache max dirty': 0}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 248, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'sleep_before_teardown': 0, 'subset': '116/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880', 'targets': {'smithi032.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGKI0SD16larEaK7qL3fVqFA11FbcmJpsb4IcefiXnbll4p4C4mcv5bUPPSto1cOjvYegynPUinhln7rMA507Ro=', 'smithi046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFX1RgxW6Y1kxefH4lcBkWNrOTP/uB2pgrtM5z5hz6SpEBSwrj+qsb6iyGQeS8ha0NU38vVAH15W+Lxh34jmx2c='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_22:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T09:16:20.349 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T09:16:20.380 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T09:16:20.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T09:16:20.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T09:16:20.444 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:20.444 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T09:16:20.487 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T09:16:20.487 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2024-09-13T09:16:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T09:16:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:16:20.536 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 875 Links: 1 2024-09-13T09:16:20.536 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:16:20.536 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-09-13 09:11:00.552172101 +0000 2024-09-13T09:16:20.536 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-09-13 09:11:00.308174280 +0000 2024-09-13T09:16:20.536 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-09-13 09:11:00.308174280 +0000 2024-09-13T09:16:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-09-13T09:16:20.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T09:16:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-09-13T09:16:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-09-13T09:16:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000274093 s, 1.9 MB/s 2024-09-13T09:16:20.591 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T09:16:20.641 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2024-09-13T09:16:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T09:16:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:16:20.687 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 899 Links: 1 2024-09-13T09:16:20.687 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:16:20.687 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-09-13 09:11:00.792169956 +0000 2024-09-13T09:16:20.688 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-09-13 09:11:00.788169993 +0000 2024-09-13T09:16:20.688 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-09-13 09:11:00.788169993 +0000 2024-09-13T09:16:20.688 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-09-13T09:16:20.688 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T09:16:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-09-13T09:16:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-09-13T09:16:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000310081 s, 1.7 MB/s 2024-09-13T09:16:20.743 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T09:16:20.792 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2024-09-13T09:16:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T09:16:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:16:20.839 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 912 Links: 1 2024-09-13T09:16:20.839 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:16:20.839 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-09-13 09:11:01.564163061 +0000 2024-09-13T09:16:20.840 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-09-13 09:11:01.308165349 +0000 2024-09-13T09:16:20.840 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-09-13 09:11:01.308165349 +0000 2024-09-13T09:16:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-09-13T09:16:20.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T09:16:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-09-13T09:16:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-09-13T09:16:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000313898 s, 1.6 MB/s 2024-09-13T09:16:20.895 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T09:16:20.945 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2024-09-13T09:16:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T09:16:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:16:20.995 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 932 Links: 1 2024-09-13T09:16:20.995 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:16:20.996 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-09-13 09:11:02.056158667 +0000 2024-09-13T09:16:20.996 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-09-13 09:11:01.792161025 +0000 2024-09-13T09:16:20.996 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-09-13 09:11:01.792161025 +0000 2024-09-13T09:16:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-09-13T09:16:20.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T09:16:21.050 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-09-13T09:16:21.050 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-09-13T09:16:21.050 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000308021 s, 1.7 MB/s 2024-09-13T09:16:21.051 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T09:16:21.101 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:16:21.101 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:21.101 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T09:16:21.107 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T09:16:21.107 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-09-13T09:16:21.155 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T09:16:21.155 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:16:21.155 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 878 Links: 1 2024-09-13T09:16:21.155 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:16:21.155 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-13 09:11:00.615336756 +0000 2024-09-13T09:16:21.155 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-13 09:11:00.335339257 +0000 2024-09-13T09:16:21.155 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-13 09:11:00.335339257 +0000 2024-09-13T09:16:21.155 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-09-13T09:16:21.155 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T09:16:21.209 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-13T09:16:21.209 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-13T09:16:21.209 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000322414 s, 1.6 MB/s 2024-09-13T09:16:21.211 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T09:16:21.260 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-09-13T09:16:21.306 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T09:16:21.306 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:16:21.306 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 898 Links: 1 2024-09-13T09:16:21.306 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:16:21.306 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-13 09:11:01.115332289 +0000 2024-09-13T09:16:21.307 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-13 09:11:00.847334684 +0000 2024-09-13T09:16:21.307 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-13 09:11:00.847334684 +0000 2024-09-13T09:16:21.307 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-09-13T09:16:21.307 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T09:16:21.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-13T09:16:21.361 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-13T09:16:21.361 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000304221 s, 1.7 MB/s 2024-09-13T09:16:21.362 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T09:16:21.412 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-09-13T09:16:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T09:16:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:16:21.459 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 915 Links: 1 2024-09-13T09:16:21.459 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:16:21.459 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-13 09:11:01.611327859 +0000 2024-09-13T09:16:21.459 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-13 09:11:01.355330146 +0000 2024-09-13T09:16:21.459 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-13 09:11:01.355330146 +0000 2024-09-13T09:16:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-09-13T09:16:21.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T09:16:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-13T09:16:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-13T09:16:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000309741 s, 1.7 MB/s 2024-09-13T09:16:21.514 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T09:16:21.564 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-09-13T09:16:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T09:16:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:16:21.611 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 933 Links: 1 2024-09-13T09:16:21.611 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:16:21.611 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-13 09:11:02.111323394 +0000 2024-09-13T09:16:21.611 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-13 09:11:01.831325894 +0000 2024-09-13T09:16:21.611 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-13 09:11:01.831325894 +0000 2024-09-13T09:16:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-09-13T09:16:21.611 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T09:16:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-13T09:16:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-13T09:16:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000302648 s, 1.7 MB/s 2024-09-13T09:16:21.667 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T09:16:21.716 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:16:21.716 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T09:16:21.716 INFO:tasks.ceph:Generating config... 2024-09-13T09:16:21.717 INFO:tasks.ceph:[client] rbd cache = True 2024-09-13T09:16:21.717 INFO:tasks.ceph:[client] rbd cache max dirty = 0 2024-09-13T09:16:21.717 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-13T09:16:21.718 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-13T09:16:21.718 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-13T09:16:21.718 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T09:16:21.718 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T09:16:21.718 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T09:16:21.718 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T09:16:21.718 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T09:16:21.718 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T09:16:21.718 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-09-13T09:16:21.718 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-13T09:16:21.718 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T09:16:21.718 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T09:16:21.718 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T09:16:21.718 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T09:16:21.719 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T09:16:21.719 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T09:16:21.719 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T09:16:21.719 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T09:16:21.719 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T09:16:21.719 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-13T09:16:21.719 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T09:16:21.719 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-13T09:16:21.719 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T09:16:21.719 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-13T09:16:21.719 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T09:16:21.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T09:16:21.755 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T09:16:21.776 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T09:16:21.856 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:16:21.910 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.32'), ('mon.c', '[v2:172.21.15.32:3301,v1:172.21.15.32:6790]'), ('mon.b', '172.21.15.46')] 2024-09-13T09:16:21.911 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],172.21.15.46', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache max dirty': 0}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-13T09:16:21.911 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:21.912 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T09:16:21.959 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.32 --addv c '[v2:172.21.15.32:3301,v1:172.21.15.32:6790]' --add b 172.21.15.46 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:22.047 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:22.047 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: generated fsid f0889cb1-03d6-4b81-88c9-6141846c6033 2024-09-13T09:16:22.047 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = pacific 2024-09-13T09:16:22.048 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 0 2024-09-13T09:16:22.048 INFO:teuthology.orchestra.run.smithi032.stdout:fsid f0889cb1-03d6-4b81-88c9-6141846c6033 2024-09-13T09:16:22.048 INFO:teuthology.orchestra.run.smithi032.stdout:last_changed 2024-09-13T09:16:22.048580+0000 2024-09-13T09:16:22.048 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-09-13T09:16:22.048580+0000 2024-09-13T09:16:22.048 INFO:teuthology.orchestra.run.smithi032.stdout:min_mon_release 16 (pacific) 2024-09-13T09:16:22.048 INFO:teuthology.orchestra.run.smithi032.stdout:election_strategy: 1 2024-09-13T09:16:22.048 INFO:teuthology.orchestra.run.smithi032.stdout:0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-09-13T09:16:22.048 INFO:teuthology.orchestra.run.smithi032.stdout:1: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.b 2024-09-13T09:16:22.048 INFO:teuthology.orchestra.run.smithi032.stdout:2: [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] mon.c 2024-09-13T09:16:22.048 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T09:16:22.049 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T09:16:22.055 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f0889cb1-03d6-4b81-88c9-6141846c6033... 2024-09-13T09:16:22.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T09:16:22.100 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T09:16:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2024-09-13T09:16:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2024-09-13T09:16:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2024-09-13T09:16:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2024-09-13T09:16:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2024-09-13T09:16:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2024-09-13T09:16:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T09:16:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T09:16:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2024-09-13T09:16:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T09:16:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2024-09-13T09:16:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],172.21.15.46" 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: mon client directed command retry = 5 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on pool no app = False 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: ms inject socket failures = 5000 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = f0889cb1-03d6-4b81-88c9-6141846c6033 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2024-09-13T09:16:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore compression algorithm = lz4 2024-09-13T09:16:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore compression mode = aggressive 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 1/20 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 1/20 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 4/10 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: osd blocked scrub grace period = 3600 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: osd max markdown count = 1000 2024-09-13T09:16:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: osd shutdown pgref assert = True 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-09-13T09:16:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T09:16:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T09:16:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: rbd cache = True 2024-09-13T09:16:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: rbd cache max dirty = 0 2024-09-13T09:16:22.140 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2024-09-13T09:16:22.140 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2024-09-13T09:16:22.140 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2024-09-13T09:16:22.140 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-09-13T09:16:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-09-13T09:16:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-09-13T09:16:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-09-13T09:16:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],172.21.15.46" 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: mon client directed command retry = 5 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on pool no app = False 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: ms inject socket failures = 5000 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = f0889cb1-03d6-4b81-88c9-6141846c6033 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-09-13T09:16:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-09-13T09:16:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore compression algorithm = lz4 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore compression mode = aggressive 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2024-09-13T09:16:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: osd blocked scrub grace period = 3600 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: osd max markdown count = 1000 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: osd shutdown pgref assert = True 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-09-13T09:16:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-09-13T09:16:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: rbd cache = True 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: rbd cache max dirty = 0 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-09-13T09:16:22.150 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-09-13T09:16:22.151 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-09-13T09:16:22.152 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T09:16:22.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T09:16:22.243 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T09:16:22.243 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:22.244 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T09:16:22.248 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:22.248 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T09:16:22.295 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2024-09-13T09:16:22.296 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:22.296 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T09:16:22.296 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:16:22.360 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:22.360 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:22.407 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi046.front.sepia.ceph.com 2024-09-13T09:16:22.407 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:22.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T09:16:22.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:16:22.431 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:22.431 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:22.478 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T09:16:22.478 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T09:16:22.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T09:16:22.524 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T09:16:22.527 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T09:16:22.571 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T09:16:22.581 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T09:16:22.582 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T09:16:22.582 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T09:16:22.619 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T09:16:22.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T09:16:22.667 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T09:16:22.677 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T09:16:22.677 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T09:16:22.678 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T09:16:22.690 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:16:22.690 INFO:tasks.ceph:role: osd.0 2024-09-13T09:16:22.691 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-13T09:16:22.691 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T09:16:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:16:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:16:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:16:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:16:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:16:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:16:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:16:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:16:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:16:22.747 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:16:22.771 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-09-13T09:16:22.772 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-09-13T09:16:22.773 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T09:16:22.833 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T09:16:22.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:22.886 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:16:22.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T09:16:22.943 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:16:22.943 INFO:tasks.ceph:role: osd.1 2024-09-13T09:16:22.943 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-13T09:16:22.944 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T09:16:22.999 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:16:22.999 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:16:22.999 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:16:23.000 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:16:23.000 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:16:23.000 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:16:23.000 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:16:23.000 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:16:23.000 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:16:23.000 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:16:23.023 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-09-13T09:16:23.025 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-09-13T09:16:23.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T09:16:23.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T09:16:23.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:23.134 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:16:23.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T09:16:23.191 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:16:23.191 INFO:tasks.ceph:role: osd.2 2024-09-13T09:16:23.191 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-13T09:16:23.192 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T09:16:23.248 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:16:23.248 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:16:23.248 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:16:23.248 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:16:23.248 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:16:23.248 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:16:23.248 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:16:23.248 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:16:23.249 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:16:23.249 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:16:23.273 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-09-13T09:16:23.275 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-09-13T09:16:23.275 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T09:16:23.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T09:16:23.389 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:16:23.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:23.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T09:16:23.442 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:16:23.442 INFO:tasks.ceph:role: osd.3 2024-09-13T09:16:23.442 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-13T09:16:23.442 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T09:16:23.499 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:16:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:16:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:16:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:16:23.499 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:16:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:16:23.500 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:16:23.500 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:16:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:16:23.500 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:16:23.526 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-09-13T09:16:23.528 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-09-13T09:16:23.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T09:16:23.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T09:16:23.639 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:16:23.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:23.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:23.753+0000 7f1d50ccd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T09:16:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:23.753+0000 7f1d50ccd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T09:16:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:23.753+0000 7f1d50ccd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T09:16:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:23.753+0000 7f1d50ccd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T09:16:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:23.753+0000 7f1d50ccd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T09:16:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:23.753+0000 7f1d50ccd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T09:16:26.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T09:16:26.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:26.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:26.801+0000 7efc835cb7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T09:16:26.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:26.801+0000 7efc835cb7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T09:16:26.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:26.801+0000 7efc835cb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T09:16:26.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:26.801+0000 7efc835cb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T09:16:26.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:26.801+0000 7efc835cb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T09:16:26.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:26.805+0000 7efc835cb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T09:16:29.805 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T09:16:29.860 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:29.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:29.941+0000 7f5e1bf117c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T09:16:29.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:29.941+0000 7f5e1bf117c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T09:16:29.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:29.941+0000 7f5e1bf117c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T09:16:29.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:29.941+0000 7f5e1bf117c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T09:16:29.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:29.941+0000 7f5e1bf117c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T09:16:29.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:29.941+0000 7f5e1bf117c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T09:16:32.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T09:16:32.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:33.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:33.061+0000 7f7b4998d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T09:16:33.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:33.061+0000 7f7b4998d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T09:16:33.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:33.061+0000 7f7b4998d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T09:16:33.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:33.061+0000 7f7b4998d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T09:16:33.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:33.061+0000 7f7b4998d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T09:16:33.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:16:33.061+0000 7f7b4998d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T09:16:36.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T09:16:36.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T09:16:36.091 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:16:36.091 INFO:tasks.ceph:role: osd.4 2024-09-13T09:16:36.091 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-13T09:16:36.092 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T09:16:36.146 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:16:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:16:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:16:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:16:36.146 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:16:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:16:36.146 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:16:36.147 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:16:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:16:36.147 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:16:36.169 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-09-13T09:16:36.171 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-13T09:16:36.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T09:16:36.231 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T09:16:36.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:36.284 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:16:36.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T09:16:36.335 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:16:36.335 INFO:tasks.ceph:role: osd.5 2024-09-13T09:16:36.336 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-13T09:16:36.336 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T09:16:36.389 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:16:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:16:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:16:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:16:36.389 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:16:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:16:36.390 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:16:36.390 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:16:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:16:36.390 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:16:36.413 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-09-13T09:16:36.415 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-13T09:16:36.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T09:16:36.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T09:16:36.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:36.527 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:16:36.527 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T09:16:36.579 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:16:36.580 INFO:tasks.ceph:role: osd.6 2024-09-13T09:16:36.580 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-13T09:16:36.580 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T09:16:36.634 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:16:36.634 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:16:36.634 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:16:36.635 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:16:36.635 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:16:36.635 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:16:36.635 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:16:36.635 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:16:36.635 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:16:36.635 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:16:36.659 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-09-13T09:16:36.661 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-13T09:16:36.661 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T09:16:36.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T09:16:36.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:16:36.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T09:16:36.837 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:16:36.837 INFO:tasks.ceph:role: osd.7 2024-09-13T09:16:36.837 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-13T09:16:36.837 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T09:16:36.894 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:16:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:16:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:16:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:16:36.895 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:16:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:16:36.895 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:16:36.895 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:16:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:16:36.895 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:16:36.918 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-09-13T09:16:36.920 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-13T09:16:36.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T09:16:36.982 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T09:16:37.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:16:37.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:37.135+0000 7f1d7ecbe7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T09:16:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:37.135+0000 7f1d7ecbe7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T09:16:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:37.135+0000 7f1d7ecbe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T09:16:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:37.135+0000 7f1d7ecbe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T09:16:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:37.135+0000 7f1d7ecbe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T09:16:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:37.135+0000 7f1d7ecbe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T09:16:40.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T09:16:40.113 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:40.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:40.191+0000 7f32318197c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T09:16:40.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:40.191+0000 7f32318197c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T09:16:40.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:40.191+0000 7f32318197c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T09:16:40.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:40.195+0000 7f32318197c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T09:16:40.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:40.195+0000 7f32318197c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T09:16:40.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:40.195+0000 7f32318197c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T09:16:43.172 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T09:16:43.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:43.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:43.307+0000 7f551f0667c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T09:16:43.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:43.307+0000 7f551f0667c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T09:16:43.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:43.307+0000 7f551f0667c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T09:16:43.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:43.307+0000 7f551f0667c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T09:16:43.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:43.307+0000 7f551f0667c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T09:16:43.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:43.307+0000 7f551f0667c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T09:16:46.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T09:16:46.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:46.415+0000 7f1c705ef7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T09:16:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:46.415+0000 7f1c705ef7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T09:16:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:46.415+0000 7f1c705ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T09:16:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:46.419+0000 7f1c705ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T09:16:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:46.419+0000 7f1c705ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T09:16:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-13T09:16:46.419+0000 7f1c705ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T09:16:49.405 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T09:16:49.463 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T09:16:49.463 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:49.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T09:16:49.478 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:49.478 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T09:16:49.538 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:49.538 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T09:16:49.595 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:49.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T09:16:49.650 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:49.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T09:16:49.707 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:49.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T09:16:49.721 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:49.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T09:16:49.778 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:49.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T09:16:49.834 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:49.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T09:16:49.890 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:49.890 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T09:16:49.946 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:49.946 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T09:16:49.952 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:49.952 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T09:16:49.994 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T09:16:49.995 DEBUG:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T09:16:49.997 DEBUG:teuthology.orchestra.run.smithi046:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T09:16:50.040 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.y] 2024-09-13T09:16:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDmAuRmNOpNHxAA334p/rD7rg5omYNq8P2jpw== 2024-09-13T09:16:50.040 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2024-09-13T09:16:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDnAuRm/H4jLRAAAsgDB0Jefy/JCjWl0FoNKQ== 2024-09-13T09:16:50.040 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2024-09-13T09:16:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDqAuRmXWUdMBAAoIfpUUpar2mAZ+kQyS+Pog== 2024-09-13T09:16:50.041 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2024-09-13T09:16:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDtAuRm6+tVOBAAG3p+7ejVPVstMwEg3dD55w== 2024-09-13T09:16:50.041 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.3] 2024-09-13T09:16:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDxAuRmQ4HZAxAADio3dmHaCrsFASy3zd0Gwg== 2024-09-13T09:16:50.041 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2024-09-13T09:16:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDmAuRm4DIKIhAAKirS+EEfQbU+O6VE5Oi7RQ== 2024-09-13T09:16:50.041 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.4] 2024-09-13T09:16:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQD1AuRm4UNHCBAA4cOhU6aabyuuDvn7wxlsVQ== 2024-09-13T09:16:50.041 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.5] 2024-09-13T09:16:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQD4AuRm8CHGCxAAIAIAfIMD09p/yDLZM7sC6Q== 2024-09-13T09:16:50.042 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.6] 2024-09-13T09:16:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQD7AuRmoQd+EhAA711QePpRdt/EUpn29Wp05A== 2024-09-13T09:16:50.042 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.7] 2024-09-13T09:16:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQD+AuRmm/wfGRAA/2x/Pr+q6754ldje9YAU1w== 2024-09-13T09:16:50.042 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-09-13T09:16:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDmAuRmv+b/JBAA5VWSPweCsGnrE0uWPKT+YQ== 2024-09-13T09:16:50.042 INFO:teuthology.orchestra.run.smithi032.stdout:[client.1] 2024-09-13T09:16:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDmAuRmz/rCJxAARRccM7UKS5WH5Dpx66WsdA== 2024-09-13T09:16:50.049 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.y] 2024-09-13T09:16:50.049 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDmAuRmNOpNHxAA334p/rD7rg5omYNq8P2jpw== 2024-09-13T09:16:50.049 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.0] 2024-09-13T09:16:50.049 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDnAuRm/H4jLRAAAsgDB0Jefy/JCjWl0FoNKQ== 2024-09-13T09:16:50.049 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.1] 2024-09-13T09:16:50.049 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDqAuRmXWUdMBAAoIfpUUpar2mAZ+kQyS+Pog== 2024-09-13T09:16:50.049 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.2] 2024-09-13T09:16:50.049 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDtAuRm6+tVOBAAG3p+7ejVPVstMwEg3dD55w== 2024-09-13T09:16:50.049 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.3] 2024-09-13T09:16:50.049 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDxAuRmQ4HZAxAADio3dmHaCrsFASy3zd0Gwg== 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.x] 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDmAuRm4DIKIhAAKirS+EEfQbU+O6VE5Oi7RQ== 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.4] 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQD1AuRm4UNHCBAA4cOhU6aabyuuDvn7wxlsVQ== 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.5] 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQD4AuRm8CHGCxAAIAIAfIMD09p/yDLZM7sC6Q== 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.6] 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQD7AuRmoQd+EhAA711QePpRdt/EUpn29Wp05A== 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.7] 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQD+AuRmm/wfGRAA/2x/Pr+q6754ldje9YAU1w== 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDmAuRmv+b/JBAA5VWSPweCsGnrE0uWPKT+YQ== 2024-09-13T09:16:50.050 INFO:teuthology.orchestra.run.smithi046.stdout:[client.1] 2024-09-13T09:16:50.051 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDmAuRmz/rCJxAARRccM7UKS5WH5Dpx66WsdA== 2024-09-13T09:16:50.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:16:50.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:16:50.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.222 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.310 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.451 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.510 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:16:50.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:16:50.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.605 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.820 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:16:50.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:16:50.957 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:16:51.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:16:51.046 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:16:51.099 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T09:16:51.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T09:16:51.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T09:16:51.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T09:16:51.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T09:16:51.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T09:16:51.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T09:16:51.411 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T09:16:51.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T09:16:51.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T09:16:51.533 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:51.535 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:16:51.577 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T09:16:51.578 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T09:16:51.578 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T09:16:51.584 INFO:tasks.ceph.mon.a:Started 2024-09-13T09:16:51.584 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T09:16:51.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T09:16:51.587 INFO:tasks.ceph.mon.c:Started 2024-09-13T09:16:51.587 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T09:16:51.587 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T09:16:51.624 INFO:tasks.ceph.mon.b:Started 2024-09-13T09:16:51.624 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T09:16:51.625 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T09:16:51.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T09:16:51.628 INFO:tasks.ceph.mgr.y:Started 2024-09-13T09:16:51.629 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T09:16:51.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T09:16:51.633 INFO:tasks.ceph.mgr.x:Started 2024-09-13T09:16:51.633 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T09:16:51.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T09:16:57.460 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:16:57.457+0000 7f096f16f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T09:16:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2024-09-13T09:16:57.869 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T09:16:57.869 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T09:16:57.870 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:57.870 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T09:16:57.880 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:57.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T09:16:57.934 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:57.934 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T09:16:57.990 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-13T09:16:57.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T09:16:58.000 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:16:57.997+0000 7f096f16f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T09:16:58.012 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:58.012 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T09:16:58.026 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:58.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T09:16:58.031 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:16:58.027+0000 7fe9722d8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T09:16:58.042 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:58.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T09:16:58.097 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-13T09:16:58.098 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T09:16:58.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 786bdfc4-cc25-4167-aa5b-3737cfeb4ac1 0 2024-09-13T09:16:58.221 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:16:58.217+0000 7f096f16f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T09:16:58.433 INFO:teuthology.orchestra.run.smithi046.stdout:0 2024-09-13T09:16:58.451 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new ca6189ab-90dd-4e9d-a85f-092186153b5a 1 2024-09-13T09:16:58.552 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:16:58.547+0000 7fe9722d8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T09:16:58.642 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-13T09:16:58.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a 2 2024-09-13T09:16:58.763 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:16:58.759+0000 7fe9722d8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T09:16:58.813 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:16:58.809+0000 7f096f16f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T09:16:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-09-13T09:16:58.869 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218 3 2024-09-13T09:16:58.870 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:16:58.865+0000 7f096f16f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T09:16:58.925 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:16:58.921+0000 7f096f16f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T09:16:59.008 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:16:59.005+0000 7f096f16f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T09:16:59.062 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:16:59.057+0000 7f096f16f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T09:16:59.066 INFO:teuthology.orchestra.run.smithi046.stdout:3 2024-09-13T09:16:59.082 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 39f42e87-29dd-42cc-9ef6-e0074f1794e4 4 2024-09-13T09:16:59.143 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:16:59.141+0000 7f096f16f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T09:16:59.197 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:16:59.193+0000 7f096f16f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T09:16:59.276 INFO:teuthology.orchestra.run.smithi046.stdout:4 2024-09-13T09:16:59.293 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new cf8dd701-9543-4bdd-a6f6-0b56210f00c2 5 2024-09-13T09:16:59.355 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:16:59.353+0000 7f096f16f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T09:16:59.383 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:16:59.379+0000 7fe9722d8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T09:16:59.419 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:16:59.417+0000 7f096f16f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T09:16:59.441 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:16:59.435+0000 7fe9722d8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T09:16:59.489 INFO:teuthology.orchestra.run.smithi046.stdout:5 2024-09-13T09:16:59.497 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:16:59.491+0000 7fe9722d8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T09:16:59.506 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new dd080b88-3292-4a34-8dbd-645683d8fe14 6 2024-09-13T09:16:59.581 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:16:59.575+0000 7fe9722d8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T09:16:59.637 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:16:59.631+0000 7fe9722d8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T09:16:59.706 INFO:teuthology.orchestra.run.smithi046.stdout:6 2024-09-13T09:16:59.719 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:16:59.715+0000 7fe9722d8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T09:16:59.722 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new c6bca54b-e982-4d6a-9ae9-057ee3881a5b 7 2024-09-13T09:16:59.777 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:16:59.771+0000 7fe9722d8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T09:16:59.916 INFO:teuthology.orchestra.run.smithi046.stdout:7 2024-09-13T09:16:59.932 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T09:16:59.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T09:16:59.935 INFO:tasks.ceph.osd.0:Started 2024-09-13T09:16:59.935 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T09:16:59.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T09:16:59.938 INFO:tasks.ceph.osd.1:Started 2024-09-13T09:16:59.938 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T09:16:59.938 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T09:16:59.938 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:16:59.935+0000 7fe9722d8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T09:16:59.941 INFO:tasks.ceph.osd.2:Started 2024-09-13T09:16:59.941 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T09:16:59.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T09:16:59.944 INFO:tasks.ceph.osd.3:Started 2024-09-13T09:16:59.945 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T09:16:59.945 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T09:16:59.948 INFO:tasks.ceph.osd.4:Started 2024-09-13T09:16:59.948 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T09:16:59.948 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T09:16:59.951 INFO:tasks.ceph.osd.5:Started 2024-09-13T09:16:59.951 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T09:16:59.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T09:16:59.954 INFO:tasks.ceph.osd.6:Started 2024-09-13T09:16:59.954 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T09:16:59.954 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T09:16:59.956 INFO:tasks.ceph.osd.7:Started 2024-09-13T09:16:59.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:17:00.017 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:17:00.011+0000 7fe9722d8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T09:17:00.069 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T09:17:00.069 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T09:17:00.069 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2024-09-13T09:17:00.111 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:00.109+0000 7f096f16f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T09:17:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:17:00.166 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"fsid":"f0889cb1-03d6-4b81-88c9-6141846c6033","created":"2024-09-13T09:16:56.964982+0000","modified":"2024-09-13T09:16:59.912481+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T09:16:59.912481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"786bdfc4-cc25-4167-aa5b-3737cfeb4ac1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ca6189ab-90dd-4e9d-a85f-092186153b5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"39f42e87-29dd-42cc-9ef6-e0074f1794e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cf8dd701-9543-4bdd-a6f6-0b56210f00c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dd080b88-3292-4a34-8dbd-645683d8fe14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c6bca54b-e982-4d6a-9ae9-057ee3881a5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:17:00.186 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T09:17:00.186 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T09:17:00.263 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:00.261+0000 7f096f16f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:17:00.319 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:00.317+0000 7f375bf687c0 -1 Falling back to public interface 2024-09-13T09:17:00.445 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:00.441+0000 7f096f16f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T09:17:00.516 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:00.513+0000 7f096f16f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T09:17:00.572 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:00.569+0000 7f1a377f77c0 -1 Falling back to public interface 2024-09-13T09:17:00.572 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:00.569+0000 7f97af88a7c0 -1 Falling back to public interface 2024-09-13T09:17:00.579 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:00.575+0000 7ff9e67517c0 -1 Falling back to public interface 2024-09-13T09:17:00.586 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:00.585+0000 7fd6fde567c0 -1 Falling back to public interface 2024-09-13T09:17:00.587 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:00.583+0000 7fefadbda7c0 -1 Falling back to public interface 2024-09-13T09:17:00.587 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:00.583+0000 7f2f7cfd47c0 -1 Falling back to public interface 2024-09-13T09:17:00.590 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:00.587+0000 7fdc6c4e17c0 -1 Falling back to public interface 2024-09-13T09:17:00.659 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:00.657+0000 7f096f16f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:17:00.663 INFO:tasks.ceph.mgr.x.smithi046.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T09:17:00.664 INFO:tasks.ceph.mgr.x.smithi046.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T09:17:00.664 INFO:tasks.ceph.mgr.x.smithi046.stderr: from numpy import show_config as show_numpy_config 2024-09-13T09:17:00.741 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:17:00.735+0000 7fe9722d8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T09:17:00.925 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:17:00.919+0000 7fe9722d8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:17:00.991 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T09:17:01.101 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:17:01.095+0000 7fe9722d8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T09:17:01.173 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:17:01.167+0000 7fe9722d8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T09:17:01.175 INFO:teuthology.misc.health.smithi032.stdout: 2024-09-13T09:17:01.175 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":10,"fsid":"f0889cb1-03d6-4b81-88c9-6141846c6033","created":"2024-09-13T09:16:56.964982+0000","modified":"2024-09-13T09:16:59.912481+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T09:16:59.912481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"786bdfc4-cc25-4167-aa5b-3737cfeb4ac1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ca6189ab-90dd-4e9d-a85f-092186153b5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"39f42e87-29dd-42cc-9ef6-e0074f1794e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cf8dd701-9543-4bdd-a6f6-0b56210f00c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dd080b88-3292-4a34-8dbd-645683d8fe14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c6bca54b-e982-4d6a-9ae9-057ee3881a5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:17:01.190 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T09:17:01.312 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:01.309+0000 7f096f16f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T09:17:01.323 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:17:01.319+0000 7fe9722d8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:17:01.391 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:01.389+0000 7f096f16f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T09:17:01.580 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:01.577+0000 7f096f16f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T09:17:01.657 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:01.653+0000 7f096f16f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T09:17:01.955 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:17:01.951+0000 7fe9722d8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T09:17:01.962 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:01.961+0000 7f096f16f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T09:17:02.028 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:02.025+0000 7f096f16f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T09:17:02.037 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:17:02.031+0000 7fe9722d8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T09:17:02.225 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:17:02.219+0000 7fe9722d8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T09:17:02.302 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:17:02.299+0000 7fe9722d8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T09:17:02.606 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:17:02.603+0000 7fe9722d8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T09:17:02.670 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-13T09:17:02.667+0000 7fe9722d8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T09:17:03.063 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:03.061+0000 7f0916722640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T09:17:03.065 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-13T09:17:03.061+0000 7f0916722640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T09:17:04.167 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:04.165+0000 7f1a377f77c0 -1 osd.1 0 log_to_monitors true 2024-09-13T09:17:04.267 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:04.265+0000 7f375bf687c0 -1 osd.0 0 log_to_monitors true 2024-09-13T09:17:04.457 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:04.451+0000 7fefadbda7c0 -1 osd.5 0 log_to_monitors true 2024-09-13T09:17:04.463 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:04.459+0000 7f2f7cfd47c0 -1 osd.6 0 log_to_monitors true 2024-09-13T09:17:04.464 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:04.461+0000 7f97af88a7c0 -1 osd.3 0 log_to_monitors true 2024-09-13T09:17:04.479 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:04.473+0000 7fd6fde567c0 -1 osd.2 0 log_to_monitors true 2024-09-13T09:17:04.493 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:04.491+0000 7fdc6c4e17c0 -1 osd.4 0 log_to_monitors true 2024-09-13T09:17:04.496 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:04.491+0000 7ff9e67517c0 -1 osd.7 0 log_to_monitors true 2024-09-13T09:17:07.324 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:07.321+0000 7f3757f0d640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T09:17:07.351 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:07.349+0000 7f374fd40640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:17:07.360 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:07.355+0000 7fefa9b7f640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T09:17:07.364 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:07.361+0000 7fd6f9dfb640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T09:17:07.376 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:07.373+0000 7f97ab82f640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T09:17:07.390 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:07.387+0000 7fefa19b2640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:17:07.391 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:07.389+0000 7f1a33fb3640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T09:17:07.394 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:07.391+0000 7fdc68c9d640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T09:17:07.397 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:07.391+0000 7f2f78f79640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T09:17:07.398 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:07.397+0000 7fd6f1c2e640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:17:07.410 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:07.409+0000 7f97a3662640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:17:07.422 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:07.419+0000 7fdc602b9640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:17:07.426 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:07.425+0000 7f1a2b5cf640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:17:07.430 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:07.427+0000 7f2f70dac640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:17:07.470 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:07.467+0000 7ff9e2f0d640 -1 osd.7 0 waiting for initial osdmap 2024-09-13T09:17:07.491 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:07.487+0000 7ff9da529640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:17:07.999 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T09:17:08.293 INFO:teuthology.misc.health.smithi032.stdout: 2024-09-13T09:17:08.293 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":12,"fsid":"f0889cb1-03d6-4b81-88c9-6141846c6033","created":"2024-09-13T09:16:56.964982+0000","modified":"2024-09-13T09:17:06.072497+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T09:16:59.912481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"786bdfc4-cc25-4167-aa5b-3737cfeb4ac1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ca6189ab-90dd-4e9d-a85f-092186153b5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"39f42e87-29dd-42cc-9ef6-e0074f1794e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cf8dd701-9543-4bdd-a6f6-0b56210f00c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dd080b88-3292-4a34-8dbd-645683d8fe14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c6bca54b-e982-4d6a-9ae9-057ee3881a5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:17:08.310 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T09:17:15.117 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T09:17:15.416 INFO:teuthology.misc.health.smithi032.stdout: 2024-09-13T09:17:15.416 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":17,"fsid":"f0889cb1-03d6-4b81-88c9-6141846c6033","created":"2024-09-13T09:16:56.964982+0000","modified":"2024-09-13T09:17:12.384182+0000","last_up_change":"2024-09-13T09:17:08.368378+0000","last_in_change":"2024-09-13T09:16:59.912481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:17:10.140920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"786bdfc4-cc25-4167-aa5b-3737cfeb4ac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6801","nonce":3549111487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6803","nonce":3549111487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6807","nonce":3549111487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6805","nonce":3549111487}]},"public_addr":"172.21.15.32:6801/3549111487","cluster_addr":"172.21.15.32:6803/3549111487","heartbeat_back_addr":"172.21.15.32:6807/3549111487","heartbeat_front_addr":"172.21.15.32:6805/3549111487","state":["exists","up"]},{"osd":1,"uuid":"ca6189ab-90dd-4e9d-a85f-092186153b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6810","nonce":1470371899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6815","nonce":1470371899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6823","nonce":1470371899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6820","nonce":1470371899}]},"public_addr":"172.21.15.32:6810/1470371899","cluster_addr":"172.21.15.32:6815/1470371899","heartbeat_back_addr":"172.21.15.32:6823/1470371899","heartbeat_front_addr":"172.21.15.32:6820/1470371899","state":["exists","up"]},{"osd":2,"uuid":"c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6825","nonce":2485123706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6827","nonce":2485123706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6831","nonce":2485123706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6829","nonce":2485123706}]},"public_addr":"172.21.15.32:6825/2485123706","cluster_addr":"172.21.15.32:6827/2485123706","heartbeat_back_addr":"172.21.15.32:6831/2485123706","heartbeat_front_addr":"172.21.15.32:6829/2485123706","state":["exists","up"]},{"osd":3,"uuid":"5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6811","nonce":1009890408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6814","nonce":1009890408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6821","nonce":1009890408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6818","nonce":1009890408}]},"public_addr":"172.21.15.32:6811/1009890408","cluster_addr":"172.21.15.32:6814/1009890408","heartbeat_back_addr":"172.21.15.32:6821/1009890408","heartbeat_front_addr":"172.21.15.32:6818/1009890408","state":["exists","up"]},{"osd":4,"uuid":"39f42e87-29dd-42cc-9ef6-e0074f1794e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6823","nonce":2723023689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6825","nonce":2723023689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6831","nonce":2723023689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6828","nonce":2723023689}]},"public_addr":"172.21.15.46:6823/2723023689","cluster_addr":"172.21.15.46:6825/2723023689","heartbeat_back_addr":"172.21.15.46:6831/2723023689","heartbeat_front_addr":"172.21.15.46:6828/2723023689","state":["exists","up"]},{"osd":5,"uuid":"cf8dd701-9543-4bdd-a6f6-0b56210f00c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6809","nonce":2955019640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6812","nonce":2955019640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2955019640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6815","nonce":2955019640}]},"public_addr":"172.21.15.46:6809/2955019640","cluster_addr":"172.21.15.46:6812/2955019640","heartbeat_back_addr":"172.21.15.46:6819/2955019640","heartbeat_front_addr":"172.21.15.46:6815/2955019640","state":["exists","up"]},{"osd":6,"uuid":"dd080b88-3292-4a34-8dbd-645683d8fe14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6813","nonce":916160151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6818","nonce":916160151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6829","nonce":916160151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6822","nonce":916160151}]},"public_addr":"172.21.15.46:6813/916160151","cluster_addr":"172.21.15.46:6818/916160151","heartbeat_back_addr":"172.21.15.46:6829/916160151","heartbeat_front_addr":"172.21.15.46:6822/916160151","state":["exists","up"]},{"osd":7,"uuid":"c6bca54b-e982-4d6a-9ae9-057ee3881a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6801","nonce":3477167689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6803","nonce":3477167689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6807","nonce":3477167689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6805","nonce":3477167689}]},"public_addr":"172.21.15.46:6801/3477167689","cluster_addr":"172.21.15.46:6803/3477167689","heartbeat_back_addr":"172.21.15.46:6807/3477167689","heartbeat_front_addr":"172.21.15.46:6805/3477167689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.247113+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.166872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.451769+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.458130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.466129+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.444828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.425145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.484371+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:17:15.432 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T09:17:15.432 INFO:tasks.ceph:Creating RBD pool 2024-09-13T09:17:15.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T09:17:16.481 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'rbd' created 2024-09-13T09:17:16.506 DEBUG:teuthology.orchestra.run.smithi032:> rbd --cluster ceph pool init rbd 2024-09-13T09:17:19.531 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T09:17:19.531 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T09:17:19.532 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T09:17:19.532 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T09:17:19.532 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T09:17:19.894 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:17:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":73244689},{"type":"v1","addr":"172.21.15.32:6833","nonce":73244689}]},"active_addr":"172.21.15.32:6833/73244689","active_change":"2024-09-13T09:17:02.049546+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2329986831}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2294319258}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1856872146}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3929286250}]}]} 2024-09-13T09:17:19.917 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T09:17:19.917 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T09:17:19.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:17:20.214 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:17:20.215 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"f0889cb1-03d6-4b81-88c9-6141846c6033","created":"2024-09-13T09:16:56.964982+0000","modified":"2024-09-13T09:17:19.495999+0000","last_up_change":"2024-09-13T09:17:08.368378+0000","last_in_change":"2024-09-13T09:16:59.912481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:17:10.140920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:17:15.775544+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"786bdfc4-cc25-4167-aa5b-3737cfeb4ac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6801","nonce":3549111487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6803","nonce":3549111487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6807","nonce":3549111487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6805","nonce":3549111487}]},"public_addr":"172.21.15.32:6801/3549111487","cluster_addr":"172.21.15.32:6803/3549111487","heartbeat_back_addr":"172.21.15.32:6807/3549111487","heartbeat_front_addr":"172.21.15.32:6805/3549111487","state":["exists","up"]},{"osd":1,"uuid":"ca6189ab-90dd-4e9d-a85f-092186153b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6810","nonce":1470371899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6815","nonce":1470371899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6823","nonce":1470371899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6820","nonce":1470371899}]},"public_addr":"172.21.15.32:6810/1470371899","cluster_addr":"172.21.15.32:6815/1470371899","heartbeat_back_addr":"172.21.15.32:6823/1470371899","heartbeat_front_addr":"172.21.15.32:6820/1470371899","state":["exists","up"]},{"osd":2,"uuid":"c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6825","nonce":2485123706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6827","nonce":2485123706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6831","nonce":2485123706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6829","nonce":2485123706}]},"public_addr":"172.21.15.32:6825/2485123706","cluster_addr":"172.21.15.32:6827/2485123706","heartbeat_back_addr":"172.21.15.32:6831/2485123706","heartbeat_front_addr":"172.21.15.32:6829/2485123706","state":["exists","up"]},{"osd":3,"uuid":"5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6811","nonce":1009890408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6814","nonce":1009890408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6821","nonce":1009890408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6818","nonce":1009890408}]},"public_addr":"172.21.15.32:6811/1009890408","cluster_addr":"172.21.15.32:6814/1009890408","heartbeat_back_addr":"172.21.15.32:6821/1009890408","heartbeat_front_addr":"172.21.15.32:6818/1009890408","state":["exists","up"]},{"osd":4,"uuid":"39f42e87-29dd-42cc-9ef6-e0074f1794e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6823","nonce":2723023689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6825","nonce":2723023689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6831","nonce":2723023689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6828","nonce":2723023689}]},"public_addr":"172.21.15.46:6823/2723023689","cluster_addr":"172.21.15.46:6825/2723023689","heartbeat_back_addr":"172.21.15.46:6831/2723023689","heartbeat_front_addr":"172.21.15.46:6828/2723023689","state":["exists","up"]},{"osd":5,"uuid":"cf8dd701-9543-4bdd-a6f6-0b56210f00c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6809","nonce":2955019640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6812","nonce":2955019640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2955019640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6815","nonce":2955019640}]},"public_addr":"172.21.15.46:6809/2955019640","cluster_addr":"172.21.15.46:6812/2955019640","heartbeat_back_addr":"172.21.15.46:6819/2955019640","heartbeat_front_addr":"172.21.15.46:6815/2955019640","state":["exists","up"]},{"osd":6,"uuid":"dd080b88-3292-4a34-8dbd-645683d8fe14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6813","nonce":916160151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6818","nonce":916160151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6829","nonce":916160151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6822","nonce":916160151}]},"public_addr":"172.21.15.46:6813/916160151","cluster_addr":"172.21.15.46:6818/916160151","heartbeat_back_addr":"172.21.15.46:6829/916160151","heartbeat_front_addr":"172.21.15.46:6822/916160151","state":["exists","up"]},{"osd":7,"uuid":"c6bca54b-e982-4d6a-9ae9-057ee3881a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6801","nonce":3477167689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6803","nonce":3477167689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6807","nonce":3477167689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6805","nonce":3477167689}]},"public_addr":"172.21.15.46:6801/3477167689","cluster_addr":"172.21.15.46:6803/3477167689","heartbeat_back_addr":"172.21.15.46:6807/3477167689","heartbeat_front_addr":"172.21.15.46:6805/3477167689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.247113+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.166872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.451769+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.458130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.466129+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.444828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.425145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.484371+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:17:20.233 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T09:17:20.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:17:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:17:20.516 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"f0889cb1-03d6-4b81-88c9-6141846c6033","created":"2024-09-13T09:16:56.964982+0000","modified":"2024-09-13T09:17:19.495999+0000","last_up_change":"2024-09-13T09:17:08.368378+0000","last_in_change":"2024-09-13T09:16:59.912481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:17:10.140920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:17:15.775544+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"786bdfc4-cc25-4167-aa5b-3737cfeb4ac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6801","nonce":3549111487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6803","nonce":3549111487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6807","nonce":3549111487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6805","nonce":3549111487}]},"public_addr":"172.21.15.32:6801/3549111487","cluster_addr":"172.21.15.32:6803/3549111487","heartbeat_back_addr":"172.21.15.32:6807/3549111487","heartbeat_front_addr":"172.21.15.32:6805/3549111487","state":["exists","up"]},{"osd":1,"uuid":"ca6189ab-90dd-4e9d-a85f-092186153b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6810","nonce":1470371899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6815","nonce":1470371899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6823","nonce":1470371899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6820","nonce":1470371899}]},"public_addr":"172.21.15.32:6810/1470371899","cluster_addr":"172.21.15.32:6815/1470371899","heartbeat_back_addr":"172.21.15.32:6823/1470371899","heartbeat_front_addr":"172.21.15.32:6820/1470371899","state":["exists","up"]},{"osd":2,"uuid":"c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6825","nonce":2485123706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6827","nonce":2485123706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6831","nonce":2485123706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6829","nonce":2485123706}]},"public_addr":"172.21.15.32:6825/2485123706","cluster_addr":"172.21.15.32:6827/2485123706","heartbeat_back_addr":"172.21.15.32:6831/2485123706","heartbeat_front_addr":"172.21.15.32:6829/2485123706","state":["exists","up"]},{"osd":3,"uuid":"5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6811","nonce":1009890408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6814","nonce":1009890408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6821","nonce":1009890408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6818","nonce":1009890408}]},"public_addr":"172.21.15.32:6811/1009890408","cluster_addr":"172.21.15.32:6814/1009890408","heartbeat_back_addr":"172.21.15.32:6821/1009890408","heartbeat_front_addr":"172.21.15.32:6818/1009890408","state":["exists","up"]},{"osd":4,"uuid":"39f42e87-29dd-42cc-9ef6-e0074f1794e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6823","nonce":2723023689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6825","nonce":2723023689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6831","nonce":2723023689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6828","nonce":2723023689}]},"public_addr":"172.21.15.46:6823/2723023689","cluster_addr":"172.21.15.46:6825/2723023689","heartbeat_back_addr":"172.21.15.46:6831/2723023689","heartbeat_front_addr":"172.21.15.46:6828/2723023689","state":["exists","up"]},{"osd":5,"uuid":"cf8dd701-9543-4bdd-a6f6-0b56210f00c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6809","nonce":2955019640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6812","nonce":2955019640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2955019640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6815","nonce":2955019640}]},"public_addr":"172.21.15.46:6809/2955019640","cluster_addr":"172.21.15.46:6812/2955019640","heartbeat_back_addr":"172.21.15.46:6819/2955019640","heartbeat_front_addr":"172.21.15.46:6815/2955019640","state":["exists","up"]},{"osd":6,"uuid":"dd080b88-3292-4a34-8dbd-645683d8fe14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6813","nonce":916160151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6818","nonce":916160151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6829","nonce":916160151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6822","nonce":916160151}]},"public_addr":"172.21.15.46:6813/916160151","cluster_addr":"172.21.15.46:6818/916160151","heartbeat_back_addr":"172.21.15.46:6829/916160151","heartbeat_front_addr":"172.21.15.46:6822/916160151","state":["exists","up"]},{"osd":7,"uuid":"c6bca54b-e982-4d6a-9ae9-057ee3881a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6801","nonce":3477167689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6803","nonce":3477167689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6807","nonce":3477167689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6805","nonce":3477167689}]},"public_addr":"172.21.15.46:6801/3477167689","cluster_addr":"172.21.15.46:6803/3477167689","heartbeat_back_addr":"172.21.15.46:6807/3477167689","heartbeat_front_addr":"172.21.15.46:6805/3477167689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.247113+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.166872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.451769+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.458130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.466129+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.444828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.425145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.484371+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:17:20.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T09:17:20.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T09:17:20.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T09:17:20.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T09:17:20.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T09:17:20.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T09:17:20.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T09:17:20.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T09:17:20.779 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-09-13T09:17:20.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:17:20.801 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-09-13T09:17:20.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:17:20.807 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-09-13T09:17:20.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:17:20.808 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-09-13T09:17:20.809 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:17:20.813 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-09-13T09:17:20.814 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:17:20.815 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2024-09-13T09:17:20.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:17:20.819 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-09-13T09:17:20.820 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:17:20.832 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2024-09-13T09:17:20.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:17:21.309 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2024-09-13T09:17:21.318 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2024-09-13T09:17:21.332 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-09-13T09:17:21.333 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-09-13T09:17:21.342 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-09-13T09:17:21.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-09-13T09:17:21.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-09-13T09:17:21.358 INFO:teuthology.orchestra.run.smithi032.stdout:55834574851 2024-09-13T09:17:21.361 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-09-13T09:17:21.367 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-09-13T09:17:21.372 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2024-09-13T09:17:21.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2024-09-13T09:17:21.391 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-09-13T09:17:21.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-09-13T09:17:21.396 INFO:teuthology.orchestra.run.smithi032.stdout:55834574851 2024-09-13T09:17:21.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.5 2024-09-13T09:17:22.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:17:22.352 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:17:22.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:17:22.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:17:22.380 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:17:22.392 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:17:22.392 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:17:22.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:17:22.848 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-09-13T09:17:22.872 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-09-13T09:17:22.872 DEBUG:teuthology.parallel:result is None 2024-09-13T09:17:22.886 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-09-13T09:17:22.894 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-09-13T09:17:22.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-09-13T09:17:22.915 DEBUG:teuthology.parallel:result is None 2024-09-13T09:17:22.919 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2024-09-13T09:17:22.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-09-13T09:17:22.922 DEBUG:teuthology.parallel:result is None 2024-09-13T09:17:22.932 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-09-13T09:17:22.937 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2024-09-13T09:17:22.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-09-13T09:17:22.938 DEBUG:teuthology.parallel:result is None 2024-09-13T09:17:22.940 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2024-09-13T09:17:22.955 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2024-09-13T09:17:22.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-09-13T09:17:22.959 DEBUG:teuthology.parallel:result is None 2024-09-13T09:17:22.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-09-13T09:17:22.963 DEBUG:teuthology.parallel:result is None 2024-09-13T09:17:22.964 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-09-13T09:17:22.964 DEBUG:teuthology.parallel:result is None 2024-09-13T09:17:22.974 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2024-09-13T09:17:22.974 DEBUG:teuthology.parallel:result is None 2024-09-13T09:17:22.974 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T09:17:22.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:17:23.296 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:17:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:17:23.312 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T09:17:22.064393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":250756,"kb_used_data":1816,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754723964,"statfs":{"total":773094113280,"available":772837339136,"internally_reserved":0,"allocated":1859584,"data_stored":1769022,"data_compressed":18678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.585127"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.574027+0000","last_change":"2024-09-13T09:17:19.574596+0000","last_active":"2024-09-13T09:17:19.574027+0000","last_peered":"2024-09-13T09:17:19.574027+0000","last_clean":"2024-09-13T09:17:19.574027+0000","last_became_active":"2024-09-13T09:17:17.495563+0000","last_became_peered":"2024-09-13T09:17:17.495563+0000","last_unstale":"2024-09-13T09:17:19.574027+0000","last_undegraded":"2024-09-13T09:17:19.574027+0000","last_fullsized":"2024-09-13T09:17:19.574027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:00.074904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096986200000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.503348+0000","last_change":"2024-09-13T09:17:19.504545+0000","last_active":"2024-09-13T09:17:19.503348+0000","last_peered":"2024-09-13T09:17:19.503348+0000","last_clean":"2024-09-13T09:17:19.503348+0000","last_became_active":"2024-09-13T09:17:17.494308+0000","last_became_peered":"2024-09-13T09:17:17.494308+0000","last_unstale":"2024-09-13T09:17:19.503348+0000","last_undegraded":"2024-09-13T09:17:19.503348+0000","last_fullsized":"2024-09-13T09:17:19.503348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:26.637575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503208,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.574777+0000","last_change":"2024-09-13T09:17:19.575344+0000","last_active":"2024-09-13T09:17:19.574777+0000","last_peered":"2024-09-13T09:17:19.574777+0000","last_clean":"2024-09-13T09:17:19.574777+0000","last_became_active":"2024-09-13T09:17:17.578605+0000","last_became_peered":"2024-09-13T09:17:17.578605+0000","last_unstale":"2024-09-13T09:17:19.574777+0000","last_undegraded":"2024-09-13T09:17:19.574777+0000","last_fullsized":"2024-09-13T09:17:19.574777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:12.007708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.503215+0000","last_change":"2024-09-13T09:17:19.503835+0000","last_active":"2024-09-13T09:17:19.503215+0000","last_peered":"2024-09-13T09:17:19.503215+0000","last_clean":"2024-09-13T09:17:19.503215+0000","last_became_active":"2024-09-13T09:17:17.580381+0000","last_became_peered":"2024-09-13T09:17:17.580381+0000","last_unstale":"2024-09-13T09:17:19.503215+0000","last_undegraded":"2024-09-13T09:17:19.503215+0000","last_fullsized":"2024-09-13T09:17:19.503215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:32.159522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.509855+0000","last_change":"2024-09-13T09:17:19.509855+0000","last_active":"2024-09-13T09:17:19.509855+0000","last_peered":"2024-09-13T09:17:19.509855+0000","last_clean":"2024-09-13T09:17:19.509855+0000","last_became_active":"2024-09-13T09:17:17.495349+0000","last_became_peered":"2024-09-13T09:17:17.495349+0000","last_unstale":"2024-09-13T09:17:19.509855+0000","last_undegraded":"2024-09-13T09:17:19.509855+0000","last_fullsized":"2024-09-13T09:17:19.509855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:26:42.881389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.575207+0000","last_change":"2024-09-13T09:17:19.575553+0000","last_active":"2024-09-13T09:17:19.575207+0000","last_peered":"2024-09-13T09:17:19.575207+0000","last_clean":"2024-09-13T09:17:19.575207+0000","last_became_active":"2024-09-13T09:17:17.493930+0000","last_became_peered":"2024-09-13T09:17:17.493930+0000","last_unstale":"2024-09-13T09:17:19.575207+0000","last_undegraded":"2024-09-13T09:17:19.575207+0000","last_fullsized":"2024-09-13T09:17:19.575207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:12.092799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000590198,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.574655+0000","last_change":"2024-09-13T09:17:19.575190+0000","last_active":"2024-09-13T09:17:19.574655+0000","last_peered":"2024-09-13T09:17:19.574655+0000","last_clean":"2024-09-13T09:17:19.574655+0000","last_became_active":"2024-09-13T09:17:17.494196+0000","last_became_peered":"2024-09-13T09:17:17.494196+0000","last_unstale":"2024-09-13T09:17:19.574655+0000","last_undegraded":"2024-09-13T09:17:19.574655+0000","last_fullsized":"2024-09-13T09:17:19.574655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:50.071449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.507800+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:17:19.507800+0000","last_peered":"2024-09-13T09:17:19.507800+0000","last_clean":"2024-09-13T09:17:19.507800+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:17:19.507800+0000","last_undegraded":"2024-09-13T09:17:19.507800+0000","last_fullsized":"2024-09-13T09:17:19.507800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T09:17:18.675918+0000","last_change":"2024-09-13T09:17:11.468790+0000","last_active":"2024-09-13T09:17:18.675918+0000","last_peered":"2024-09-13T09:17:18.675918+0000","last_clean":"2024-09-13T09:17:18.675918+0000","last_became_active":"2024-09-13T09:17:11.466046+0000","last_became_peered":"2024-09-13T09:17:11.466046+0000","last_unstale":"2024-09-13T09:17:18.675918+0000","last_undegraded":"2024-09-13T09:17:18.675918+0000","last_fullsized":"2024-09-13T09:17:18.675918+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:40:02.916796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":274432,"data_stored":663899,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":221184,"data_stored":73550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":221184,"data_stored":73550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":274432,"data_stored":663899,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:17:23.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:17:23.596 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:17:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:17:23.612 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T09:17:22.064393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":250756,"kb_used_data":1816,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754723964,"statfs":{"total":773094113280,"available":772837339136,"internally_reserved":0,"allocated":1859584,"data_stored":1769022,"data_compressed":18678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.585127"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.574027+0000","last_change":"2024-09-13T09:17:19.574596+0000","last_active":"2024-09-13T09:17:19.574027+0000","last_peered":"2024-09-13T09:17:19.574027+0000","last_clean":"2024-09-13T09:17:19.574027+0000","last_became_active":"2024-09-13T09:17:17.495563+0000","last_became_peered":"2024-09-13T09:17:17.495563+0000","last_unstale":"2024-09-13T09:17:19.574027+0000","last_undegraded":"2024-09-13T09:17:19.574027+0000","last_fullsized":"2024-09-13T09:17:19.574027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:00.074904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096986200000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.503348+0000","last_change":"2024-09-13T09:17:19.504545+0000","last_active":"2024-09-13T09:17:19.503348+0000","last_peered":"2024-09-13T09:17:19.503348+0000","last_clean":"2024-09-13T09:17:19.503348+0000","last_became_active":"2024-09-13T09:17:17.494308+0000","last_became_peered":"2024-09-13T09:17:17.494308+0000","last_unstale":"2024-09-13T09:17:19.503348+0000","last_undegraded":"2024-09-13T09:17:19.503348+0000","last_fullsized":"2024-09-13T09:17:19.503348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:26.637575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503208,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.574777+0000","last_change":"2024-09-13T09:17:19.575344+0000","last_active":"2024-09-13T09:17:19.574777+0000","last_peered":"2024-09-13T09:17:19.574777+0000","last_clean":"2024-09-13T09:17:19.574777+0000","last_became_active":"2024-09-13T09:17:17.578605+0000","last_became_peered":"2024-09-13T09:17:17.578605+0000","last_unstale":"2024-09-13T09:17:19.574777+0000","last_undegraded":"2024-09-13T09:17:19.574777+0000","last_fullsized":"2024-09-13T09:17:19.574777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:12.007708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.503215+0000","last_change":"2024-09-13T09:17:19.503835+0000","last_active":"2024-09-13T09:17:19.503215+0000","last_peered":"2024-09-13T09:17:19.503215+0000","last_clean":"2024-09-13T09:17:19.503215+0000","last_became_active":"2024-09-13T09:17:17.580381+0000","last_became_peered":"2024-09-13T09:17:17.580381+0000","last_unstale":"2024-09-13T09:17:19.503215+0000","last_undegraded":"2024-09-13T09:17:19.503215+0000","last_fullsized":"2024-09-13T09:17:19.503215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:32.159522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.509855+0000","last_change":"2024-09-13T09:17:19.509855+0000","last_active":"2024-09-13T09:17:19.509855+0000","last_peered":"2024-09-13T09:17:19.509855+0000","last_clean":"2024-09-13T09:17:19.509855+0000","last_became_active":"2024-09-13T09:17:17.495349+0000","last_became_peered":"2024-09-13T09:17:17.495349+0000","last_unstale":"2024-09-13T09:17:19.509855+0000","last_undegraded":"2024-09-13T09:17:19.509855+0000","last_fullsized":"2024-09-13T09:17:19.509855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:26:42.881389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.575207+0000","last_change":"2024-09-13T09:17:19.575553+0000","last_active":"2024-09-13T09:17:19.575207+0000","last_peered":"2024-09-13T09:17:19.575207+0000","last_clean":"2024-09-13T09:17:19.575207+0000","last_became_active":"2024-09-13T09:17:17.493930+0000","last_became_peered":"2024-09-13T09:17:17.493930+0000","last_unstale":"2024-09-13T09:17:19.575207+0000","last_undegraded":"2024-09-13T09:17:19.575207+0000","last_fullsized":"2024-09-13T09:17:19.575207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:12.092799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000590198,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.574655+0000","last_change":"2024-09-13T09:17:19.575190+0000","last_active":"2024-09-13T09:17:19.574655+0000","last_peered":"2024-09-13T09:17:19.574655+0000","last_clean":"2024-09-13T09:17:19.574655+0000","last_became_active":"2024-09-13T09:17:17.494196+0000","last_became_peered":"2024-09-13T09:17:17.494196+0000","last_unstale":"2024-09-13T09:17:19.574655+0000","last_undegraded":"2024-09-13T09:17:19.574655+0000","last_fullsized":"2024-09-13T09:17:19.574655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:50.071449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.507800+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:17:19.507800+0000","last_peered":"2024-09-13T09:17:19.507800+0000","last_clean":"2024-09-13T09:17:19.507800+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:17:19.507800+0000","last_undegraded":"2024-09-13T09:17:19.507800+0000","last_fullsized":"2024-09-13T09:17:19.507800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T09:17:18.675918+0000","last_change":"2024-09-13T09:17:11.468790+0000","last_active":"2024-09-13T09:17:18.675918+0000","last_peered":"2024-09-13T09:17:18.675918+0000","last_clean":"2024-09-13T09:17:18.675918+0000","last_became_active":"2024-09-13T09:17:11.466046+0000","last_became_peered":"2024-09-13T09:17:11.466046+0000","last_unstale":"2024-09-13T09:17:18.675918+0000","last_undegraded":"2024-09-13T09:17:18.675918+0000","last_fullsized":"2024-09-13T09:17:18.675918+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:40:02.916796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":274432,"data_stored":663899,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":221184,"data_stored":73550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":221184,"data_stored":73550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":274432,"data_stored":663899,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:17:23.614 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T09:17:23.614 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T09:17:23.614 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T09:17:23.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T09:17:23.932 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:17:23.933 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T09:17:23.950 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T09:17:23.950 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-13T09:17:23.964 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-13T09:17:23.965 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache max dirty': 0}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}} 2024-09-13T09:17:23.965 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-13T09:17:23.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-13T09:17:24.284 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-13T09:17:24.284 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T09:17:24.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:17:24.568 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:17:24.568 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:17:24.584 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-13T09:17:24.065012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":250756,"kb_used_data":1816,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754723964,"statfs":{"total":773094113280,"available":772837339136,"internally_reserved":0,"allocated":1859584,"data_stored":1769022,"data_compressed":18678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.583141"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.574027+0000","last_change":"2024-09-13T09:17:19.574596+0000","last_active":"2024-09-13T09:17:19.574027+0000","last_peered":"2024-09-13T09:17:19.574027+0000","last_clean":"2024-09-13T09:17:19.574027+0000","last_became_active":"2024-09-13T09:17:17.495563+0000","last_became_peered":"2024-09-13T09:17:17.495563+0000","last_unstale":"2024-09-13T09:17:19.574027+0000","last_undegraded":"2024-09-13T09:17:19.574027+0000","last_fullsized":"2024-09-13T09:17:19.574027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:00.074904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096986200000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.503348+0000","last_change":"2024-09-13T09:17:19.504545+0000","last_active":"2024-09-13T09:17:19.503348+0000","last_peered":"2024-09-13T09:17:19.503348+0000","last_clean":"2024-09-13T09:17:19.503348+0000","last_became_active":"2024-09-13T09:17:17.494308+0000","last_became_peered":"2024-09-13T09:17:17.494308+0000","last_unstale":"2024-09-13T09:17:19.503348+0000","last_undegraded":"2024-09-13T09:17:19.503348+0000","last_fullsized":"2024-09-13T09:17:19.503348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:26.637575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503208,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.574777+0000","last_change":"2024-09-13T09:17:19.575344+0000","last_active":"2024-09-13T09:17:19.574777+0000","last_peered":"2024-09-13T09:17:19.574777+0000","last_clean":"2024-09-13T09:17:19.574777+0000","last_became_active":"2024-09-13T09:17:17.578605+0000","last_became_peered":"2024-09-13T09:17:17.578605+0000","last_unstale":"2024-09-13T09:17:19.574777+0000","last_undegraded":"2024-09-13T09:17:19.574777+0000","last_fullsized":"2024-09-13T09:17:19.574777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:12.007708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.503215+0000","last_change":"2024-09-13T09:17:19.503835+0000","last_active":"2024-09-13T09:17:19.503215+0000","last_peered":"2024-09-13T09:17:19.503215+0000","last_clean":"2024-09-13T09:17:19.503215+0000","last_became_active":"2024-09-13T09:17:17.580381+0000","last_became_peered":"2024-09-13T09:17:17.580381+0000","last_unstale":"2024-09-13T09:17:19.503215+0000","last_undegraded":"2024-09-13T09:17:19.503215+0000","last_fullsized":"2024-09-13T09:17:19.503215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:32.159522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.509855+0000","last_change":"2024-09-13T09:17:19.509855+0000","last_active":"2024-09-13T09:17:19.509855+0000","last_peered":"2024-09-13T09:17:19.509855+0000","last_clean":"2024-09-13T09:17:19.509855+0000","last_became_active":"2024-09-13T09:17:17.495349+0000","last_became_peered":"2024-09-13T09:17:17.495349+0000","last_unstale":"2024-09-13T09:17:19.509855+0000","last_undegraded":"2024-09-13T09:17:19.509855+0000","last_fullsized":"2024-09-13T09:17:19.509855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:26:42.881389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.575207+0000","last_change":"2024-09-13T09:17:19.575553+0000","last_active":"2024-09-13T09:17:19.575207+0000","last_peered":"2024-09-13T09:17:19.575207+0000","last_clean":"2024-09-13T09:17:19.575207+0000","last_became_active":"2024-09-13T09:17:17.493930+0000","last_became_peered":"2024-09-13T09:17:17.493930+0000","last_unstale":"2024-09-13T09:17:19.575207+0000","last_undegraded":"2024-09-13T09:17:19.575207+0000","last_fullsized":"2024-09-13T09:17:19.575207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:12.092799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000590198,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.574655+0000","last_change":"2024-09-13T09:17:19.575190+0000","last_active":"2024-09-13T09:17:19.574655+0000","last_peered":"2024-09-13T09:17:19.574655+0000","last_clean":"2024-09-13T09:17:19.574655+0000","last_became_active":"2024-09-13T09:17:17.494196+0000","last_became_peered":"2024-09-13T09:17:17.494196+0000","last_unstale":"2024-09-13T09:17:19.574655+0000","last_undegraded":"2024-09-13T09:17:19.574655+0000","last_fullsized":"2024-09-13T09:17:19.574655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:50.071449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.507800+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:17:19.507800+0000","last_peered":"2024-09-13T09:17:19.507800+0000","last_clean":"2024-09-13T09:17:19.507800+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:17:19.507800+0000","last_undegraded":"2024-09-13T09:17:19.507800+0000","last_fullsized":"2024-09-13T09:17:19.507800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T09:17:18.675918+0000","last_change":"2024-09-13T09:17:11.468790+0000","last_active":"2024-09-13T09:17:18.675918+0000","last_peered":"2024-09-13T09:17:18.675918+0000","last_clean":"2024-09-13T09:17:18.675918+0000","last_became_active":"2024-09-13T09:17:11.466046+0000","last_became_peered":"2024-09-13T09:17:11.466046+0000","last_unstale":"2024-09-13T09:17:18.675918+0000","last_undegraded":"2024-09-13T09:17:18.675918+0000","last_fullsized":"2024-09-13T09:17:18.675918+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:40:02.916796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":274432,"data_stored":663899,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":221184,"data_stored":73550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":221184,"data_stored":73550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":274432,"data_stored":663899,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:17:24.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:17:24.881 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:17:24.881 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:17:24.899 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-13T09:17:24.065012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":250756,"kb_used_data":1816,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754723964,"statfs":{"total":773094113280,"available":772837339136,"internally_reserved":0,"allocated":1859584,"data_stored":1769022,"data_compressed":18678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.583141"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.574027+0000","last_change":"2024-09-13T09:17:19.574596+0000","last_active":"2024-09-13T09:17:19.574027+0000","last_peered":"2024-09-13T09:17:19.574027+0000","last_clean":"2024-09-13T09:17:19.574027+0000","last_became_active":"2024-09-13T09:17:17.495563+0000","last_became_peered":"2024-09-13T09:17:17.495563+0000","last_unstale":"2024-09-13T09:17:19.574027+0000","last_undegraded":"2024-09-13T09:17:19.574027+0000","last_fullsized":"2024-09-13T09:17:19.574027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:00.074904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096986200000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.503348+0000","last_change":"2024-09-13T09:17:19.504545+0000","last_active":"2024-09-13T09:17:19.503348+0000","last_peered":"2024-09-13T09:17:19.503348+0000","last_clean":"2024-09-13T09:17:19.503348+0000","last_became_active":"2024-09-13T09:17:17.494308+0000","last_became_peered":"2024-09-13T09:17:17.494308+0000","last_unstale":"2024-09-13T09:17:19.503348+0000","last_undegraded":"2024-09-13T09:17:19.503348+0000","last_fullsized":"2024-09-13T09:17:19.503348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:26.637575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503208,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.574777+0000","last_change":"2024-09-13T09:17:19.575344+0000","last_active":"2024-09-13T09:17:19.574777+0000","last_peered":"2024-09-13T09:17:19.574777+0000","last_clean":"2024-09-13T09:17:19.574777+0000","last_became_active":"2024-09-13T09:17:17.578605+0000","last_became_peered":"2024-09-13T09:17:17.578605+0000","last_unstale":"2024-09-13T09:17:19.574777+0000","last_undegraded":"2024-09-13T09:17:19.574777+0000","last_fullsized":"2024-09-13T09:17:19.574777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:12.007708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.503215+0000","last_change":"2024-09-13T09:17:19.503835+0000","last_active":"2024-09-13T09:17:19.503215+0000","last_peered":"2024-09-13T09:17:19.503215+0000","last_clean":"2024-09-13T09:17:19.503215+0000","last_became_active":"2024-09-13T09:17:17.580381+0000","last_became_peered":"2024-09-13T09:17:17.580381+0000","last_unstale":"2024-09-13T09:17:19.503215+0000","last_undegraded":"2024-09-13T09:17:19.503215+0000","last_fullsized":"2024-09-13T09:17:19.503215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:32.159522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.509855+0000","last_change":"2024-09-13T09:17:19.509855+0000","last_active":"2024-09-13T09:17:19.509855+0000","last_peered":"2024-09-13T09:17:19.509855+0000","last_clean":"2024-09-13T09:17:19.509855+0000","last_became_active":"2024-09-13T09:17:17.495349+0000","last_became_peered":"2024-09-13T09:17:17.495349+0000","last_unstale":"2024-09-13T09:17:19.509855+0000","last_undegraded":"2024-09-13T09:17:19.509855+0000","last_fullsized":"2024-09-13T09:17:19.509855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:26:42.881389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.575207+0000","last_change":"2024-09-13T09:17:19.575553+0000","last_active":"2024-09-13T09:17:19.575207+0000","last_peered":"2024-09-13T09:17:19.575207+0000","last_clean":"2024-09-13T09:17:19.575207+0000","last_became_active":"2024-09-13T09:17:17.493930+0000","last_became_peered":"2024-09-13T09:17:17.493930+0000","last_unstale":"2024-09-13T09:17:19.575207+0000","last_undegraded":"2024-09-13T09:17:19.575207+0000","last_fullsized":"2024-09-13T09:17:19.575207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:12.092799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000590198,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.574655+0000","last_change":"2024-09-13T09:17:19.575190+0000","last_active":"2024-09-13T09:17:19.574655+0000","last_peered":"2024-09-13T09:17:19.574655+0000","last_clean":"2024-09-13T09:17:19.574655+0000","last_became_active":"2024-09-13T09:17:17.494196+0000","last_became_peered":"2024-09-13T09:17:17.494196+0000","last_unstale":"2024-09-13T09:17:19.574655+0000","last_undegraded":"2024-09-13T09:17:19.574655+0000","last_fullsized":"2024-09-13T09:17:19.574655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:50.071449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:17:19.507800+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:17:19.507800+0000","last_peered":"2024-09-13T09:17:19.507800+0000","last_clean":"2024-09-13T09:17:19.507800+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:17:19.507800+0000","last_undegraded":"2024-09-13T09:17:19.507800+0000","last_fullsized":"2024-09-13T09:17:19.507800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T09:17:18.675918+0000","last_change":"2024-09-13T09:17:11.468790+0000","last_active":"2024-09-13T09:17:18.675918+0000","last_peered":"2024-09-13T09:17:18.675918+0000","last_clean":"2024-09-13T09:17:18.675918+0000","last_became_active":"2024-09-13T09:17:11.466046+0000","last_became_peered":"2024-09-13T09:17:11.466046+0000","last_unstale":"2024-09-13T09:17:18.675918+0000","last_undegraded":"2024-09-13T09:17:18.675918+0000","last_fullsized":"2024-09-13T09:17:18.675918+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:40:02.916796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":274432,"data_stored":663899,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":221184,"data_stored":73550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":217088,"data_stored":73531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":221184,"data_stored":73550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":274432,"data_stored":663899,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:17:24.900 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T09:17:24.900 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-13T09:17:25.190 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 21 2024-09-13T09:17:25.190 INFO:teuthology.orchestra.run.smithi032.stdout:fsid f0889cb1-03d6-4b81-88c9-6141846c6033 2024-09-13T09:17:25.191 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-09-13T09:16:56.964982+0000 2024-09-13T09:17:25.191 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-09-13T09:17:19.495999+0000 2024-09-13T09:17:25.191 INFO:teuthology.orchestra.run.smithi032.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T09:17:25.191 INFO:teuthology.orchestra.run.smithi032.stdout:crush_version 4 2024-09-13T09:17:25.191 INFO:teuthology.orchestra.run.smithi032.stdout:full_ratio 0.95 2024-09-13T09:17:25.191 INFO:teuthology.orchestra.run.smithi032.stdout:backfillfull_ratio 0.9 2024-09-13T09:17:25.191 INFO:teuthology.orchestra.run.smithi032.stdout:nearfull_ratio 0.85 2024-09-13T09:17:25.191 INFO:teuthology.orchestra.run.smithi032.stdout:require_min_compat_client luminous 2024-09-13T09:17:25.191 INFO:teuthology.orchestra.run.smithi032.stdout:min_compat_client jewel 2024-09-13T09:17:25.191 INFO:teuthology.orchestra.run.smithi032.stdout:require_osd_release reef 2024-09-13T09:17:25.192 INFO:teuthology.orchestra.run.smithi032.stdout:stretch_mode_enabled false 2024-09-13T09:17:25.192 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-13T09:17:25.192 INFO:teuthology.orchestra.run.smithi032.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-13T09:17:25.192 INFO:teuthology.orchestra.run.smithi032.stdout: removed_snaps_queue [2~1] 2024-09-13T09:17:25.192 INFO:teuthology.orchestra.run.smithi032.stdout:max_osd 8 2024-09-13T09:17:25.192 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6800/3549111487,v1:172.21.15.32:6801/3549111487] [v2:172.21.15.32:6802/3549111487,v1:172.21.15.32:6803/3549111487] exists,up 786bdfc4-cc25-4167-aa5b-3737cfeb4ac1 2024-09-13T09:17:25.192 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6808/1470371899,v1:172.21.15.32:6810/1470371899] [v2:172.21.15.32:6812/1470371899,v1:172.21.15.32:6815/1470371899] exists,up ca6189ab-90dd-4e9d-a85f-092186153b5a 2024-09-13T09:17:25.192 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6824/2485123706,v1:172.21.15.32:6825/2485123706] [v2:172.21.15.32:6826/2485123706,v1:172.21.15.32:6827/2485123706] exists,up c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a 2024-09-13T09:17:25.192 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6809/1009890408,v1:172.21.15.32:6811/1009890408] [v2:172.21.15.32:6813/1009890408,v1:172.21.15.32:6814/1009890408] exists,up 5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218 2024-09-13T09:17:25.192 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6821/2723023689,v1:172.21.15.46:6823/2723023689] [v2:172.21.15.46:6824/2723023689,v1:172.21.15.46:6825/2723023689] exists,up 39f42e87-29dd-42cc-9ef6-e0074f1794e4 2024-09-13T09:17:25.193 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6808/2955019640,v1:172.21.15.46:6809/2955019640] [v2:172.21.15.46:6811/2955019640,v1:172.21.15.46:6812/2955019640] exists,up cf8dd701-9543-4bdd-a6f6-0b56210f00c2 2024-09-13T09:17:25.193 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6810/916160151,v1:172.21.15.46:6813/916160151] [v2:172.21.15.46:6816/916160151,v1:172.21.15.46:6818/916160151] exists,up dd080b88-3292-4a34-8dbd-645683d8fe14 2024-09-13T09:17:25.193 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6800/3477167689,v1:172.21.15.46:6801/3477167689] [v2:172.21.15.46:6802/3477167689,v1:172.21.15.46:6803/3477167689] exists,up c6bca54b-e982-4d6a-9ae9-057ee3881a5b 2024-09-13T09:17:25.207 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6800/3549111487,v1:172.21.15.32:6801/3549111487] [v2:172.21.15.32:6802/3549111487,v1:172.21.15.32:6803/3549111487] exists,up 786bdfc4-cc25-4167-aa5b-3737cfeb4ac1', 'osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6808/1470371899,v1:172.21.15.32:6810/1470371899] [v2:172.21.15.32:6812/1470371899,v1:172.21.15.32:6815/1470371899] exists,up ca6189ab-90dd-4e9d-a85f-092186153b5a', 'osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6824/2485123706,v1:172.21.15.32:6825/2485123706] [v2:172.21.15.32:6826/2485123706,v1:172.21.15.32:6827/2485123706] exists,up c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6809/1009890408,v1:172.21.15.32:6811/1009890408] [v2:172.21.15.32:6813/1009890408,v1:172.21.15.32:6814/1009890408] exists,up 5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6821/2723023689,v1:172.21.15.46:6823/2723023689] [v2:172.21.15.46:6824/2723023689,v1:172.21.15.46:6825/2723023689] exists,up 39f42e87-29dd-42cc-9ef6-e0074f1794e4', 'osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6808/2955019640,v1:172.21.15.46:6809/2955019640] [v2:172.21.15.46:6811/2955019640,v1:172.21.15.46:6812/2955019640] exists,up cf8dd701-9543-4bdd-a6f6-0b56210f00c2', 'osd.6 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6810/916160151,v1:172.21.15.46:6813/916160151] [v2:172.21.15.46:6816/916160151,v1:172.21.15.46:6818/916160151] exists,up dd080b88-3292-4a34-8dbd-645683d8fe14', 'osd.7 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6800/3477167689,v1:172.21.15.46:6801/3477167689] [v2:172.21.15.46:6802/3477167689,v1:172.21.15.46:6803/3477167689] exists,up c6bca54b-e982-4d6a-9ae9-057ee3881a5b'] 2024-09-13T09:17:25.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-13T09:17:25.327 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:25.327 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mon.a", 2024-09-13T09:17:25.327 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster": "ceph", 2024-09-13T09:17:25.327 INFO:teuthology.orchestra.run.smithi032.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-13T09:17:25.327 INFO:teuthology.orchestra.run.smithi032.stdout: "admin_socket_mode": "", 2024-09-13T09:17:25.328 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-13T09:17:25.328 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_client_required": "cephx, none", 2024-09-13T09:17:25.328 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_cluster_required": "cephx", 2024-09-13T09:17:25.328 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_debug": "true", 2024-09-13T09:17:25.328 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-13T09:17:25.328 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-13T09:17:25.328 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_service_required": "cephx", 2024-09-13T09:17:25.329 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-13T09:17:25.329 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_supported": "cephx", 2024-09-13T09:17:25.329 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_aio": "true", 2024-09-13T09:17:25.329 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-13T09:17:25.329 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_aio_poll_ms": "250", 2024-09-13T09:17:25.329 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_aio_reap_max": "16", 2024-09-13T09:17:25.329 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_async_discard": "false", 2024-09-13T09:17:25.329 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_block_size": "4096", 2024-09-13T09:17:25.329 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_debug_aio": "false", 2024-09-13T09:17:25.329 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-13T09:17:25.329 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-13T09:17:25.330 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_debug_inflight_ios": "false", 2024-09-13T09:17:25.330 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_enable_discard": "false", 2024-09-13T09:17:25.330 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_flock_retry": "3", 2024-09-13T09:17:25.330 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-13T09:17:25.330 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_inject_crash": "0", 2024-09-13T09:17:25.330 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-13T09:17:25.330 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_ioring": "false", 2024-09-13T09:17:25.330 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_ioring_hipri": "false", 2024-09-13T09:17:25.330 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-13T09:17:25.330 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-13T09:17:25.330 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_type": "", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_alloc_size": "1048576", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_allocator": "hybrid", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_buffered_io": "true", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_check_for_zeros": "false", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_compact_log_sync": "false", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-13T09:17:25.331 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_max_log_runway": "4194304", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_max_prefetch": "1048576", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_min_flush_size": "524288", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_min_log_runway": "1048576", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_replay_recovery": "false", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_sync_write": "false", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_allocation_from_file": "true", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_allocator": "hybrid", 2024-09-13T09:17:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-13T09:17:25.333 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-13T09:17:25.333 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-13T09:17:25.333 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-13T09:17:25.333 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-13T09:17:25.333 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-13T09:17:25.333 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-13T09:17:25.333 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_create": "true", 2024-09-13T09:17:25.333 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_db_create": "false", 2024-09-13T09:17:25.333 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_db_path": "", 2024-09-13T09:17:25.333 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_db_size": "0", 2024-09-13T09:17:25.333 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_path": "", 2024-09-13T09:17:25.333 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_preallocate_file": "false", 2024-09-13T09:17:25.334 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_size": "107374182400", 2024-09-13T09:17:25.334 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_wal_create": "false", 2024-09-13T09:17:25.334 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_wal_path": "", 2024-09-13T09:17:25.334 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_wal_size": "100663296", 2024-09-13T09:17:25.334 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bluefs": "true", 2024-09-13T09:17:25.334 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-13T09:17:25.334 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-13T09:17:25.334 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-13T09:17:25.334 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-13T09:17:25.334 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T09:17:25.334 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T09:17:25.334 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_autotune": "true", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_size": "0", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_type": "2q", 2024-09-13T09:17:25.335 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-13T09:17:25.336 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_clone_cow": "true", 2024-09-13T09:17:25.336 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-13T09:17:25.336 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-13T09:17:25.336 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-13T09:17:25.336 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-13T09:17:25.336 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-13T09:17:25.336 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-13T09:17:25.336 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-13T09:17:25.336 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_mode": "none", 2024-09-13T09:17:25.336 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-13T09:17:25.336 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_csum_type": "crc32c", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_freelist": "false", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-13T09:17:25.337 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_small_allocations": "0", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_default_buffered_read": "true", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_default_buffered_write": "false", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-13T09:17:25.338 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fail_eio": "false", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_mount": "false", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_umount": "false", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-13T09:17:25.339 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-13T09:17:25.340 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-13T09:17:25.340 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-13T09:17:25.340 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-13T09:17:25.340 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-13T09:17:25.340 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-13T09:17:25.340 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-13T09:17:25.340 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_ignore_data_csum": "false", 2024-09-13T09:17:25.340 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-13T09:17:25.340 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-13T09:17:25.340 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-13T09:17:25.340 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-13T09:17:25.340 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_log_op_age": "5.000000", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_alloc_size": "0", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_blob_size": "0", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_deferred_txc": "32", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_min_alloc_size": "0", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_nid_prealloc": "1024", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-13T09:17:25.341 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_retry_disk_reads": "3", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_rocksdb_cf": "true", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_spdk_mem": "512", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-13T09:17:25.342 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_tracing": "false", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-13T09:17:25.343 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_zero_block_detection": "false", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-13T09:17:25.344 INFO:teuthology.orchestra.run.smithi032.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-13T09:17:25.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_cluster_require_signatures": "false", 2024-09-13T09:17:25.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_cluster_require_version": "2", 2024-09-13T09:17:25.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_require_signatures": "false", 2024-09-13T09:17:25.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_require_version": "2", 2024-09-13T09:17:25.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_service_require_signatures": "false", 2024-09-13T09:17:25.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_service_require_version": "2", 2024-09-13T09:17:25.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_sign_messages": "true", 2024-09-13T09:17:25.345 INFO:teuthology.orchestra.run.smithi032.stdout: "chdir": "", 2024-09-13T09:17:25.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_acl_type": "", 2024-09-13T09:17:25.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_asio_thread_count": "2", 2024-09-13T09:17:25.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_cache_mid": "0.750000", 2024-09-13T09:17:25.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_cache_size": "16384", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_caps_release_delay": "5", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_check_pool_perm": "true", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_debug_force_sync_read": "false", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_debug_getattr_caps": "false", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_debug_inject_features": "", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_debug_inject_tick_delay": "0", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_die_on_failed_remount": "false", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_dirsize_rbytes": "true", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_force_lazyio": "false", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_fs": "", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-13T09:17:25.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_inject_release_failure": "false", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_max_inline_size": "4096", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mds_namespace": "", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_metadata": "", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mount_gid": "-1", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mount_timeout": "300", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mount_uid": "-1", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mountpoint": "/", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_notify_timeout": "10", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc": "true", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_max_dirty": "104857600", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-13T09:17:25.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_max_objects": "1000", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_size": "209715200", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_target_dirty": "8388608", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_permissions": "true", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_quota": "true", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_quota_df": "true", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_readahead_max_bytes": "0", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_readahead_max_periods": "4", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_readahead_min": "131072", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_reconnect_stale": "false", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_shutdown_timeout": "30", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_snapdir": ".snap", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_tick_interval": "1", 2024-09-13T09:17:25.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_trace": "", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_try_dentry_invalidate": "false", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_use_faked_inos": "false", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_use_random_mds": "false", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_graylog": "false", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_graylog_port": "12201", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_monitors": "default=true", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_syslog": "false", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_syslog_level": "info", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "-", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_network": "", 2024-09-13T09:17:25.349 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_network_interface": "", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "compressor_zlib_isal": "false", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "compressor_zlib_level": "5", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "compressor_zlib_winsize": "-15", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "compressor_zstd_level": "1", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_alien_op_num_threads": "6", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_seastar_smp": "1", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_location": "", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_location_hook": "", 2024-09-13T09:17:25.350 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_location_hook_timeout": "10", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "daemonize": "false", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "daos_pool": "tank", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_alienstore": "0/5", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_allow_any_pool_priority": "false", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_asok": "1/5", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_asok_assert_abort": "false", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_asserts_on_shutdown": "true", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_auth": "20/20", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_bdev": "1/3", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_bluefs": "1/5", 2024-09-13T09:17:25.351 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_bluestore": "1/5", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_buffer": "0/1", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_ceph_exporter": "1/5", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_cephfs_mirror": "0/5", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_cephsqlite": "0/5", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_client": "0/5", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_compressor": "1/5", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_context": "0/1", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_crush": "1/1", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_crypto": "1/5", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_cyanstore": "0/5", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_deliberately_leak_memory": "false", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_disable_randomized_ping": "false", 2024-09-13T09:17:25.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_dpdk": "1/5", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_eventtrace": "1/5", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_filer": "0/1", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_filestore": "1/3", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_finisher": "1/1", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_fuse": "1/5", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_heartbeat_testing_span": "0", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_heartbeatmap": "1/5", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_javaclient": "1/5", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_journal": "1/3", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_journaler": "0/5", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_kstore": "1/5", 2024-09-13T09:17:25.353 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_leveldb": "4/5", 2024-09-13T09:17:25.354 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_lockdep": "0/1", 2024-09-13T09:17:25.354 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mclock": "1/5", 2024-09-13T09:17:25.354 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds": "1/5", 2024-09-13T09:17:25.354 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_balancer": "1/5", 2024-09-13T09:17:25.354 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_locker": "1/5", 2024-09-13T09:17:25.354 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_log": "1/5", 2024-09-13T09:17:25.354 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_log_expire": "1/5", 2024-09-13T09:17:25.354 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_migrator": "1/5", 2024-09-13T09:17:25.354 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_memstore": "1/5", 2024-09-13T09:17:25.354 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mgr": "2/5", 2024-09-13T09:17:25.354 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mgrc": "1/5", 2024-09-13T09:17:25.354 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mon": "20/20", 2024-09-13T09:17:25.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_monc": "0/10", 2024-09-13T09:17:25.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_ms": "1/1", 2024-09-13T09:17:25.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_none": "0/5", 2024-09-13T09:17:25.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_objclass": "0/5", 2024-09-13T09:17:25.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_objectcacher": "0/5", 2024-09-13T09:17:25.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_objecter": "0/1", 2024-09-13T09:17:25.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_optracker": "0/5", 2024-09-13T09:17:25.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_osd": "1/5", 2024-09-13T09:17:25.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_paxos": "20/20", 2024-09-13T09:17:25.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_perfcounter": "1/5", 2024-09-13T09:17:25.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_prioritycache": "1/5", 2024-09-13T09:17:25.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rados": "0/5", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rbd": "0/5", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rbd_mirror": "0/5", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rbd_pwl": "0/5", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rbd_replay": "0/5", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_refs": "0/0", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_reserver": "1/1", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw": "1/5", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_access": "1/5", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_datacache": "1/5", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_dbstore": "1/5", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_flight": "1/5", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_sync": "1/5", 2024-09-13T09:17:25.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rocksdb": "4/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_backref": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_cache": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_cleaner": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_device": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_epm": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_journal": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_lba": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_odata": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_omap": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_onode": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_t": "0/5", 2024-09-13T09:17:25.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_tm": "0/5", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_striper": "0/1", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_test": "0/5", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_throttle": "1/1", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_timer": "0/1", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_tp": "0/5", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "device_failure_prediction_mode": "none", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "err_to_graylog": "false", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "err_to_journald": "false", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "err_to_stderr": "true", 2024-09-13T09:17:25.358 INFO:teuthology.orchestra.run.smithi032.stdout: "err_to_syslog": "false", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event_tracing": "false", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_addr": "0.0.0.0", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_http_port": "9926", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_prio_limit": "5", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_sort_metrics": "true", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_stats_period": "5", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "fake_statfs_for_testing": "0", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "fatal_signal_handlers": "true", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "filer_max_purge_ops": "10", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "filer_max_truncate_ops": "128", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_apply_finisher_threads": "1", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_blackhole": "false", 2024-09-13T09:17:25.359 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_btrfs_clone_range": "true", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_btrfs_snap": "true", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_caller_concurrency": "10", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_collect_device_partition_information": "true", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_commit_timeout": "600.000000", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_debug_inject_read_err": "false", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_debug_omap_check": "false", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_debug_verify_split": "false", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_dump_file": "", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fadvise": "true", 2024-09-13T09:17:25.360 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fail_eio": "true", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fd_cache_shards": "16", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fd_cache_size": "128", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fiemap": "false", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fiemap_threshold": "4096", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_inject_stall": "0", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_journal_parallel": "false", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_journal_trailing": "false", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_journal_writeahead": "false", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_kill_at": "0", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-13T09:17:25.361 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattrs": "0", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_xattr_value_size": "0", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_merge_threshold": "-10", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_odsync_write": "false", 2024-09-13T09:17:25.362 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_omap_backend": "rocksdb", 2024-09-13T09:17:25.363 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_omap_backend_path": "", 2024-09-13T09:17:25.363 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-13T09:17:25.363 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-13T09:17:25.363 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-13T09:17:25.363 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_op_thread_timeout": "60", 2024-09-13T09:17:25.363 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_op_threads": "2", 2024-09-13T09:17:25.363 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_punch_hole": "false", 2024-09-13T09:17:25.363 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-13T09:17:25.363 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-13T09:17:25.363 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-13T09:17:25.363 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-13T09:17:25.363 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-13T09:17:25.364 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-13T09:17:25.364 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-13T09:17:25.364 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-13T09:17:25.364 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-13T09:17:25.364 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_ops": "50", 2024-09-13T09:17:25.364 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-13T09:17:25.364 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_seek_data_hole": "false", 2024-09-13T09:17:25.364 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_sloppy_crc": "false", 2024-09-13T09:17:25.364 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-13T09:17:25.364 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_splice": "false", 2024-09-13T09:17:25.364 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_split_multiple": "2", 2024-09-13T09:17:25.364 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_split_rand_factor": "20", 2024-09-13T09:17:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_update_to": "1000", 2024-09-13T09:17:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-13T09:17:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-13T09:17:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-13T09:17:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-13T09:17:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-13T09:17:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-13T09:17:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_enable": "true", 2024-09-13T09:17:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-13T09:17:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-13T09:17:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-13T09:17:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_xfs_extsize": "false", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_zfs_snap": "false", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "fio_dir": "/tmp/fio", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "f0889cb1-03d6-4b81-88c9-6141846c6033", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_allow_other": "true", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_atomic_o_trunc": "true", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_big_writes": "true", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_debug": "false", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_default_permissions": "false", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_disable_pagecache": "false", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_max_write": "0", 2024-09-13T09:17:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_multithreaded": "true", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_require_active_mds": "true", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_set_user_groups": "true", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_splice_move": "true", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_splice_read": "true", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_splice_write": "true", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_use_invalidate_cb": "true", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "gss_target_name": "ceph", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_file": "", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_inject_failure": "0", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_interval": "5", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "host": "smithi032", 2024-09-13T09:17:25.367 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-13T09:17:25.368 INFO:teuthology.orchestra.run.smithi032.stdout: "inject_early_sigterm": "false", 2024-09-13T09:17:25.369 INFO:teuthology.orchestra.run.smithi032.stdout: "jaeger_agent_port": "6799", 2024-09-13T09:17:25.369 INFO:teuthology.orchestra.run.smithi032.stdout: "jaeger_tracing_enable": "false", 2024-09-13T09:17:25.369 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_aio": "true", 2024-09-13T09:17:25.369 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_align_min_size": "65536", 2024-09-13T09:17:25.369 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_block_align": "true", 2024-09-13T09:17:25.369 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_block_size": "4096", 2024-09-13T09:17:25.369 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_dio": "true", 2024-09-13T09:17:25.369 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_discard": "false", 2024-09-13T09:17:25.369 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_force_aio": "false", 2024-09-13T09:17:25.369 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_ignore_corruption": "false", 2024-09-13T09:17:25.369 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_max_write_bytes": "10485760", 2024-09-13T09:17:25.369 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_max_write_entries": "100", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_replay_from": "0", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_write_header_frequency": "0", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_zero_on_create": "false", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "journaler_prefetch_periods": "10", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "journaler_prezero_periods": "5", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "journaler_write_head_interval": "15", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "keyfile": "", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_backend": "rocksdb", 2024-09-13T09:17:25.370 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_default_stripe_size": "65536", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_fsck_on_mount": "false", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_max_bytes": "67108864", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_max_ops": "512", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_nid_prealloc": "1024", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_onode_map_size": "1024", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_sync_submit_transaction": "false", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_sync_transaction": "false", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_block_size": "65536", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_bloom_size": "0", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_cache_size": "536870912", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_compact_on_mount": "false", 2024-09-13T09:17:25.371 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_compression": "false", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_log": "", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_max_open_files": "0", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_paranoid": "false", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "librados_thread_count": "2", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "lockdep": "false", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "lockdep_force_backtrace": "false", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "log_coarse_timestamps": "true", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "log_flush_on_exit": "false", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "log_graylog_host": "127.0.0.1", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "log_graylog_port": "12201", 2024-09-13T09:17:25.372 INFO:teuthology.orchestra.run.smithi032.stdout: "log_max_new": "1000", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "log_max_recent": "10000", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "log_stderr_prefix": "", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "log_stop_at_utilization": "0.970000", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": "true", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_graylog": "false", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_journald": "false", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_stderr": "false", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_syslog": "false", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "max_rotating_auth_attempts": "10", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_action_on_write_error": "1", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_alternate_name_max": "8192", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_asio_thread_count": "2", 2024-09-13T09:17:25.373 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_export_pin": "true", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_fragment_dirs": "true", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_fragment_interval": "5", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_interval": "10", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_max": "-1", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_max_until": "-1", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_merge_size": "50", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_midchunk": "0.300000", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_min_start": "0.200000", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_minchunk": "0.001000", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_mode": "0", 2024-09-13T09:17:25.374 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_need_max": "1.200000", 2024-09-13T09:17:25.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_need_min": "0.800000", 2024-09-13T09:17:25.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-13T09:17:25.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-13T09:17:25.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_split_bits": "3", 2024-09-13T09:17:25.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-13T09:17:25.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_split_size": "10000", 2024-09-13T09:17:25.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-13T09:17:25.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_target_decay": "10.000000", 2024-09-13T09:17:25.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-13T09:17:25.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_beacon_grace": "15.000000", 2024-09-13T09:17:25.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_beacon_interval": "4.000000", 2024-09-13T09:17:25.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_mid": "0.700000", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_release_free_interval": "10", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_reservation": "0.050000", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_trim_interval": "1", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_trim_threshold": "262144", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_client_prealloc_inos": "1000", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-13T09:17:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_connect_bootstrapping": "false", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_damage_table_max_entries": "10000", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_debug_auth_pins": "false", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_debug_frag": "false", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_debug_scatterstat": "false", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_debug_subtrees": "false", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_decay_halflife": "5.000000", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_default_dir_hash": "2", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_defer_session_stale": "true", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_deny_all_reconnect": "false", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dir_keys_per_op": "16384", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dir_max_commit_size": "10", 2024-09-13T09:17:25.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dir_max_entries": "0", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dir_prefetch": "true", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dump_cache_on_map": "false", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_early_reply": "true", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_enable_op_tracker": "true", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_enforce_unique_name": "true", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_export_ephemeral_random": "true", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-13T09:17:25.378 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_health_summarize_threshold": "10", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_health_dummy": "false", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_migrator_session_race": "false", 2024-09-13T09:17:25.379 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_join_fs": "", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_journal_format": "1", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_create_at": "0", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_export_at": "0", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_import_at": "0", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_journal_at": "0", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_journal_expire_at": "0", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_journal_replay_at": "0", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_link_at": "0", 2024-09-13T09:17:25.380 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_mdstable_at": "0", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_openc_at": "0", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_rename_at": "0", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_events_per_segment": "1024", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_max_events": "-1", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_max_segments": "128", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_pause": "false", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_segment_size": "0", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_warn_factor": "2.000000", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_caps_per_client": "1048576", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_completed_flushes": "100000", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_completed_requests": "100000", 2024-09-13T09:17:25.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_export_size": "20971520", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_file_recover": "32", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_purge_files": "64", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_purge_ops": "8192", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_snaps_per_dir": "100", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_metrics_update_interval": "2", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_min_caps_per_client": "100", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_min_caps_working_set": "10000", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_numa_node": "-1", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_complaint_time": "30.000000", 2024-09-13T09:17:25.382 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_history_duration": "600", 2024-09-13T09:17:25.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_history_size": "20", 2024-09-13T09:17:25.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_history_slow_op_size": "20", 2024-09-13T09:17:25.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-13T09:17:25.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_log_threshold": "5", 2024-09-13T09:17:25.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_ping_grace": "15", 2024-09-13T09:17:25.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_ping_interval": "5", 2024-09-13T09:17:25.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-13T09:17:25.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-13T09:17:25.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_max_caps": "30000", 2024-09-13T09:17:25.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-13T09:17:25.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-13T09:17:25.383 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_warning_threshold": "262144", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_replay_interval": "1.000000", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_root_ino_gid": "0", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_root_ino_uid": "0", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-13T09:17:25.384 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_shutdown_check": "0", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_skip_ino": "0", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_snap_max_uid": "4294967294", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_snap_min_uid": "0", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_snap_rstat": "false", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_standby_replay_damaged": "false", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_symlink_recovery": "true", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_thrash_exports": "0", 2024-09-13T09:17:25.385 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_thrash_fragments": "0", 2024-09-13T09:17:25.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_tick_interval": "5.000000", 2024-09-13T09:17:25.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_valgrind_exit": "false", 2024-09-13T09:17:25.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_verify_backtrace": "1", 2024-09-13T09:17:25.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_verify_scatter": "false", 2024-09-13T09:17:25.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-13T09:17:25.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_wipe_sessions": "false", 2024-09-13T09:17:25.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool_debug": "false", 2024-09-13T09:17:25.386 INFO:teuthology.orchestra.run.smithi032.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-13T09:17:25.386 INFO:teuthology.orchestra.run.smithi032.stdout: "memstore_device_bytes": "1073741824", 2024-09-13T09:17:25.386 INFO:teuthology.orchestra.run.smithi032.stdout: "memstore_page_set": "false", 2024-09-13T09:17:25.386 INFO:teuthology.orchestra.run.smithi032.stdout: "memstore_page_size": "65536", 2024-09-13T09:17:25.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_client_bytes": "134217728", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_client_messages": "512", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_disabled_modules": "", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_max_pg_num_change": "128", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_mds_bytes": "134217728", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_mds_messages": "128", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_mon_bytes": "134217728", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_mon_messages": "128", 2024-09-13T09:17:25.387 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_osd_bytes": "536870912", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_osd_messages": "8192", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_pool": "true", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_standby_modules": "true", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_stats_period": "5", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_stats_threshold": "5", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_tick_period": "2", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_allow_pool_delete": "true", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_allow_pool_size_one": "true", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_auth_validate_all_caps": "true", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-13T09:17:25.388 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_bytes": "104857600", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_directed_command_retry": "5", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_parallel": "3", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_log_interval": "1.000000", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_ping_interval": "10.000000", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-13T09:17:25.389 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_file_level": "debug", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_file": "true", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_journald": "false", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_compact_on_bootstrap": "false", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_compact_on_start": "false", 2024-09-13T09:17:25.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_compact_on_trim": "true", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cpu_threads": "4", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_crush_min_required_version": "hammer", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_daemon_bytes": "419430400", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_data_avail_crit": "5", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_data_avail_warn": "5", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_data_size_warn": "16106127360", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_dump_json": "false", 2024-09-13T09:17:25.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_dump_transactions": "false", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_extra_checks": "false", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_no_require_quincy": "false", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_no_require_reef": "false", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_down_mkfs_grace": "120", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_election_default_strategy": "1", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_election_timeout": "5.000000", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-13T09:17:25.392 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_elector_ping_divisor": "2", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_enable_op_tracker": "true", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_fake_pool_delete": "false", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_force_quorum_join": "false", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_globalid_prealloc": "10000", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_detail_to_clog": "false", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_log_update_period": "5", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_max_detail": "50", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_to_clog": "true", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_to_clog_interval": "600", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_host": "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],172.21.15.46", 2024-09-13T09:17:25.393 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_host_override": "", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_initial_members": "", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_lease": "5.000000", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_log_full_interval": "50", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_log_max": "10000", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_log_max_summary": "50", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_log_epochs": "500", 2024-09-13T09:17:25.394 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_osd": "10000", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_pg_per_osd": "10000", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_pool_pg_num": "65536", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mds_force_trim_to": "0", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mds_skip_sanity": "false", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_memory_autotune": "true", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_memory_target": "2147483648", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_beacon_grace": "30", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-13T09:17:25.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_digest_period": "5", 2024-09-13T09:17:25.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_inactive_grace": "60", 2024-09-13T09:17:25.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-13T09:17:25.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-13T09:17:25.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_min_osdmap_epochs": "500", 2024-09-13T09:17:25.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_complaint_time": "30", 2024-09-13T09:17:25.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_history_duration": "600", 2024-09-13T09:17:25.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_history_size": "20", 2024-09-13T09:17:25.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_history_slow_op_size": "20", 2024-09-13T09:17:25.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-13T09:17:25.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_log_threshold": "5", 2024-09-13T09:17:25.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-13T09:17:25.396 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-13T09:17:25.397 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-13T09:17:25.397 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_auto_mark_in": "false", 2024-09-13T09:17:25.397 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-13T09:17:25.397 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-13T09:17:25.397 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-13T09:17:25.397 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_cache_size": "500", 2024-09-13T09:17:25.397 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-13T09:17:25.397 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-13T09:17:25.397 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-13T09:17:25.397 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_down_out_interval": "600", 2024-09-13T09:17:25.397 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-13T09:17:25.397 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_force_trim_to": "0", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_min_down_reporters": "2", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-13T09:17:25.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_report_timeout": "900", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-13T09:17:25.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_stuck_threshold": "60", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_probe_timeout": "2.000000", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_reweight_max_change": "0.050000", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_reweight_max_osds": "4", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-13T09:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_interval": "86400", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_max_keys": "100", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_timeout": "300", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_session_timeout": "300", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_smart_report_timeout": "5", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stat_smooth_intervals": "6", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stretch_pool_min_size": "2", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stretch_pool_size": "4", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-13T09:17:25.401 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_debug": "false", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_provider_kill_at": "0", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_requester_kill_at": "0", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_timeout": "60.000000", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_target_pg_per_osd": "100", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_tick_interval": "5", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_timecheck_interval": "300.000000", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_use_min_delay_socket": "false", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_older_version_delay": "604800", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-13T09:17:25.402 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_misplaced": "false", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_older_version": "true", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-13T09:17:25.403 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "monmap": "", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_tracing_enabled": "false", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_op_threads": "3", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-13T09:17:25.404 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_cm": "false", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_device_name": "", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_dscp": "96", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_local_gid": "", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_port_num": "1", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_sl": "3", 2024-09-13T09:17:25.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_support_srq": "true", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_type": "ib", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_reap_threshold": "5", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_before_connect": "false", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_ipv4": "true", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_ipv6": "false", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_msgr1": "true", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_msgr2": "true", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_port_max": "7568", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_port_min": "6800", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_retry_count": "3", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_retry_delay": "5", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_client": "false", 2024-09-13T09:17:25.406 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_mds": "false", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_mgr": "false", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_mon": "false", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_osd": "false", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_client_mode": "crc secure", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_cluster_mode": "crc secure", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_cluster_type": "", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_compress_secure": "false", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_connection_idle_timeout": "900", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_connection_ready_timeout": "10", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_crc_data": "true", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_crc_header": "true", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_bad_msg": "false", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_bug": "true", 2024-09-13T09:17:25.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_old_message": "true", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_skipped_message": "false", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_coremask": "0xF", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_enable_tso": "true", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_hugepages": "", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_lro": "true", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_memory_channel": "4", 2024-09-13T09:17:25.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-13T09:17:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_pmd": "", 2024-09-13T09:17:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_port_id": "0", 2024-09-13T09:17:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-13T09:17:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-13T09:17:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dump_on_send": "false", 2024-09-13T09:17:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_initial_backoff": "0.200000", 2024-09-13T09:17:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_delay_max": "1.000000", 2024-09-13T09:17:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-13T09:17:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_delay_type": "", 2024-09-13T09:17:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-13T09:17:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_network_congestion": "0", 2024-09-13T09:17:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_socket_failures": "5000", 2024-09-13T09:17:25.410 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_learn_addr_from_peer": "true", 2024-09-13T09:17:25.410 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_max_accept_failures": "4", 2024-09-13T09:17:25.410 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_max_backoff": "15.000000", 2024-09-13T09:17:25.410 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_mon_client_mode": "secure crc", 2024-09-13T09:17:25.410 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-13T09:17:25.410 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_mon_service_mode": "secure crc", 2024-09-13T09:17:25.410 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_osd_compress_min_size": "1024", 2024-09-13T09:17:25.410 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_osd_compress_mode": "none", 2024-09-13T09:17:25.410 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-13T09:17:25.411 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-13T09:17:25.411 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_pq_min_cost": "65536", 2024-09-13T09:17:25.411 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_public_type": "", 2024-09-13T09:17:25.411 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_service_mode": "crc secure", 2024-09-13T09:17:25.411 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_tcp_listen_backlog": "512", 2024-09-13T09:17:25.411 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_tcp_nodelay": "true", 2024-09-13T09:17:25.411 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-13T09:17:25.411 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_tcp_rcvbuf": "0", 2024-09-13T09:17:25.411 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_type": "async+posix", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "no_config_file": "false", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_completion_locks_per_session": "32", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_inflight_ops": "1024", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_tick_interval": "5.000000", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_timeout": "10.000000", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "objectstore_blackhole": "false", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "openssl_engine_opts": "", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_delay_time": "5.000000", 2024-09-13T09:17:25.412 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_hist_halflife": "1000", 2024-09-13T09:17:25.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_max_low_ops": "2", 2024-09-13T09:17:25.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_max_ops": "4", 2024-09-13T09:17:25.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-13T09:17:25.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-13T09:17:25.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_slop": "0.020000", 2024-09-13T09:17:25.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-13T09:17:25.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-13T09:17:25.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_asio_thread_count": "2", 2024-09-13T09:17:25.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_async_recovery_min_cost": "100", 2024-09-13T09:17:25.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-13T09:17:25.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-13T09:17:25.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backfill_scan_max": "512", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backfill_scan_min": "64", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backoff_on_degraded": "false", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backoff_on_peering": "false", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backoff_on_unfound": "true", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_beacon_report_interval": "300", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bench_duration": "30", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bench_max_block_size": "67108864", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_blkin_trace_all": "false", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-13T09:17:25.414 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_check_for_log_corruption": "false", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_class_update_on_start": "true", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_client_message_cap": "256", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_client_message_size_cap": "524288000", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_client_op_priority": "63", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_client_watch_timeout": "30", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_command_max_records": "256", 2024-09-13T09:17:25.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_compact_on_start": "false", 2024-09-13T09:17:25.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-13T09:17:25.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-13T09:17:25.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-13T09:17:25.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_crush_update_on_start": "true", 2024-09-13T09:17:25.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_crush_update_weight_set": "true", 2024-09-13T09:17:25.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-13T09:17:25.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-13T09:17:25.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-13T09:17:25.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-13T09:17:25.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-13T09:17:25.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_feed_pullee": "-1", 2024-09-13T09:17:25.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-13T09:17:25.417 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-13T09:17:25.417 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-13T09:17:25.417 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_misdirected_ops": "false", 2024-09-13T09:17:25.417 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_no_acting_change": "false", 2024-09-13T09:17:25.417 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_no_purge_strays": "false", 2024-09-13T09:17:25.417 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_op_order": "false", 2024-09-13T09:17:25.417 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-13T09:17:25.417 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-13T09:17:25.417 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-13T09:17:25.417 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-13T09:17:25.417 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_shutdown": "false", 2024-09-13T09:17:25.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-13T09:17:25.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-13T09:17:25.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-13T09:17:25.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-13T09:17:25.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_verify_snaps": "false", 2024-09-13T09:17:25.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-13T09:17:25.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-13T09:17:25.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_keys": "1024", 2024-09-13T09:17:25.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-13T09:17:25.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-13T09:17:25.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-13T09:17:25.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_stride": "524288", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_default_notify_timeout": "30", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_delete_sleep": "0.000000", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_discard_disconnected_ops": "true", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_enable_op_tracker": "true", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-13T09:17:25.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_info": "true", 2024-09-13T09:17:25.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_shutdown": "true", 2024-09-13T09:17:25.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-13T09:17:25.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-13T09:17:25.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-13T09:17:25.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-13T09:17:25.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-13T09:17:25.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_function_tracing": "false", 2024-09-13T09:17:25.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_grace": "20", 2024-09-13T09:17:25.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_interval": "6", 2024-09-13T09:17:25.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-13T09:17:25.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_min_peers": "10", 2024-09-13T09:17:25.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_min_size": "2000", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_stale": "600", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_hit_set_max_size": "100000", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_hit_set_min_size": "1000", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_journal_size": "5120", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_kill_backfill_at": "0", 2024-09-13T09:17:25.421 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_cache_size": "50", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_dedup": "true", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_message_max": "40", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_share_max_epochs": "40", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_attr_name_len": "100", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_attr_size": "0", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_backfills": "1", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_markdown_count": "5", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_markdown_period": "600", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_object_name_len": "2048", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_object_namespace_len": "256", 2024-09-13T09:17:25.422 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_object_size": "134217728", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_pg_blocked_by": "16", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_pg_log_entries": "10000", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_pgls": "1024", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_push_cost": "8388608", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_push_objects": "10", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_scrubs": "3", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_trimming_pgs": "2", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_write_op_reply_len": "64", 2024-09-13T09:17:25.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_write_size": "90", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_profile": "balanced", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-13T09:17:25.424 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_base": "805306368", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_cache_min": "134217728", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_target": "4294967296", 2024-09-13T09:17:25.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_target_autotune": "false", 2024-09-13T09:17:25.426 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-13T09:17:25.426 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_min_pg_log_entries": "250", 2024-09-13T09:17:25.426 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_min_recovery_priority": "0", 2024-09-13T09:17:25.426 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-13T09:17:25.426 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-13T09:17:25.426 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_report_interval": "5", 2024-09-13T09:17:25.426 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-13T09:17:25.426 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-13T09:17:25.426 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_num_cache_shards": "32", 2024-09-13T09:17:25.426 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_num_op_tracker_shard": "32", 2024-09-13T09:17:25.426 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_numa_auto_affinity": "true", 2024-09-13T09:17:25.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_numa_node": "-1", 2024-09-13T09:17:25.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_numa_prefer_iface": "true", 2024-09-13T09:17:25.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-13T09:17:25.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_objecter_finishers": "1", 2024-09-13T09:17:25.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_objectstore": "bluestore", 2024-09-13T09:17:25.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_objectstore_fuse": "false", 2024-09-13T09:17:25.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_objectstore_tracing": "false", 2024-09-13T09:17:25.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_complaint_time": "30.000000", 2024-09-13T09:17:25.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_history_duration": "600", 2024-09-13T09:17:25.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_history_size": "20", 2024-09-13T09:17:25.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_history_slow_op_size": "20", 2024-09-13T09:17:25.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-13T09:17:25.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_log_threshold": "5", 2024-09-13T09:17:25.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_shards": "0", 2024-09-13T09:17:25.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_shards_hdd": "5", 2024-09-13T09:17:25.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_shards_ssd": "8", 2024-09-13T09:17:25.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-13T09:17:25.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-13T09:17:25.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-13T09:17:25.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-13T09:17:25.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_pq_min_cost": "65536", 2024-09-13T09:17:25.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-13T09:17:25.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_queue_cut_off": "high", 2024-09-13T09:17:25.429 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-13T09:17:25.429 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_thread_timeout": "15", 2024-09-13T09:17:25.429 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_open_classes_on_start": "true", 2024-09-13T09:17:25.429 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_os_flags": "0", 2024-09-13T09:17:25.429 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_peering_op_priority": "255", 2024-09-13T09:17:25.429 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_delete_cost": "1048576", 2024-09-13T09:17:25.429 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_delete_priority": "5", 2024-09-13T09:17:25.429 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-13T09:17:25.429 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-13T09:17:25.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-13T09:17:25.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_log_trim_max": "10000", 2024-09-13T09:17:25.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_log_trim_min": "100", 2024-09-13T09:17:25.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-13T09:17:25.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-13T09:17:25.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-13T09:17:25.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-13T09:17:25.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-13T09:17:25.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-13T09:17:25.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-13T09:17:25.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-13T09:17:25.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_crimson": "false", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flags": "0", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_min_size": "0", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_pg_num": "32", 2024-09-13T09:17:25.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_pgp_num": "0", 2024-09-13T09:17:25.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-13T09:17:25.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_size": "2", 2024-09-13T09:17:25.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_type": "replicated", 2024-09-13T09:17:25.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-13T09:17:25.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-13T09:17:25.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_push_per_object_cost": "1000", 2024-09-13T09:17:25.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-13T09:17:25.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recover_clone_overlap": "true", 2024-09-13T09:17:25.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-13T09:17:25.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_cost": "20971520", 2024-09-13T09:17:25.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-13T09:17:25.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_active": "0", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_single_start": "1", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_op_priority": "3", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_priority": "5", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_sleep": "0.000000", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-13T09:17:25.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_repair_during_recovery": "false", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_requested_scrub_priority": "120", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_auto_repair": "false", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_begin_hour": "0", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_begin_week_day": "0", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_chunk_max": "25", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_chunk_min": "5", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_cost": "52428800", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_during_recovery": "false", 2024-09-13T09:17:25.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_end_hour": "0", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_end_week_day": "0", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_event_cost": "4096", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_invalid_stats": "true", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_max_preemptions": "5", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_priority": "5", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_sleep": "0.000000", 2024-09-13T09:17:25.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_skip_data_digest": "false", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_smart_report_timeout": "5", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_cost": "1048576", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_priority": "5", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-13T09:17:25.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_target_transaction_size": "30", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-13T09:17:25.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tracing": "false", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_use_stale_snap": "false", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "osdc_blkin_trace_all": "false", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_kill_at": "0", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_max_join_drift": "10", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_min": "500", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_min_wait": "0.050000", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_propose_interval": "1.000000", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_service_trim_max": "500", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_service_trim_min": "250", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_stash_full_interval": "25", 2024-09-13T09:17:25.438 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_trim_max": "500", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_trim_min": "250", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "perf": "true", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "-", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrv": "", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "public_bind_addr": "-", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "public_network": "", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "public_network_interface": "", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "qat_compressor_enabled": "false", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "qat_compressor_session_max_number": "256", 2024-09-13T09:17:25.439 INFO:teuthology.orchestra.run.smithi032.stdout: "rados_mon_op_timeout": "0", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rados_osd_op_timeout": "0", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rados_tracing": "false", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_atime_update_interval": "60", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_balance_parent_reads": "false", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_balance_snap_reads": "false", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_blkin_trace_all": "false", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache": "true", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-13T09:17:25.440 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_policy": "writearound", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_size": "33554432", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_clone_copy_on_read": "false", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_compression_hint": "none", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_concurrent_management_ops": "10", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_clone_format": "auto", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_data_pool": "", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_features": "61", 2024-09-13T09:17:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_format": "2", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_map_options": "", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_order": "22", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_pool": "rbd", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_stripe_count": "0", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_stripe_unit": "0", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_enable_alloc_hint": "true", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_io_scheduler": "simple", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-13T09:17:25.442 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-13T09:17:25.443 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-13T09:17:25.443 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-13T09:17:25.443 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-13T09:17:25.443 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-13T09:17:25.443 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-13T09:17:25.443 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-13T09:17:25.443 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-13T09:17:25.443 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_order": "24", 2024-09-13T09:17:25.443 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_pool": "", 2024-09-13T09:17:25.443 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_splay_width": "4", 2024-09-13T09:17:25.443 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_localize_parent_reads": "false", 2024-09-13T09:17:25.443 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_localize_snap_reads": "false", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-13T09:17:25.444 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-13T09:17:25.445 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mtime_update_interval": "60", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_non_blocking_aio": "true", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_op_thread_timeout": "60", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_op_threads": "1", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_parent_cache_enabled": "false", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-13T09:17:25.446 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_plugins": "", 2024-09-13T09:17:25.447 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_bps_burst": "0", 2024-09-13T09:17:25.447 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-13T09:17:25.447 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_bps_limit": "0", 2024-09-13T09:17:25.447 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_exclude_ops": "0", 2024-09-13T09:17:25.447 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_iops_burst": "0", 2024-09-13T09:17:25.447 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-13T09:17:25.447 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_iops_limit": "0", 2024-09-13T09:17:25.447 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-13T09:17:25.447 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-13T09:17:25.447 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-13T09:17:25.447 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-13T09:17:25.447 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_read_from_replica_policy": "default", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-13T09:17:25.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_skip_partial_discard": "true", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_tracing": "false", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_validate_names": "true", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_validate_pool": "true", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "restapi_base_url": "", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "restapi_log_level": "", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_acl_grants_max_num": "100", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_admin_entry": "admin", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_backend_store": "rados", 2024-09-13T09:17:25.449 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_barbican_url": "", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_beast_enable_async": "true", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cache_enabled": "true", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cache_expiry_interval": "900", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cache_lru_size": "10000", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_config_store": "rados", 2024-09-13T09:17:25.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_content_length_compat": "false", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_copy_obj_progress": "true", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cors_rules_max_num": "100", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cross_domain_policy": "", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_addr": "", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_password": "", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-13T09:17:25.451 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_username": "", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_require_ssl": "true", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-13T09:17:25.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-13T09:17:25.453 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-13T09:17:25.453 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-13T09:17:25.453 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-13T09:17:25.453 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_addr": "", 2024-09-13T09:17:25.453 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_auth": "token", 2024-09-13T09:17:25.453 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_namespace": "", 2024-09-13T09:17:25.453 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_prefix": "", 2024-09-13T09:17:25.453 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-13T09:17:25.453 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-13T09:17:25.453 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-13T09:17:25.453 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-13T09:17:25.453 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_token_file": "", 2024-09-13T09:17:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-13T09:17:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_buffersize": "524288", 2024-09-13T09:17:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-13T09:17:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_low_speed_time": "300", 2024-09-13T09:17:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-13T09:17:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-13T09:17:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-13T09:17:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-13T09:17:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-13T09:17:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-13T09:17:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-13T09:17:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-13T09:17:25.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-13T09:17:25.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-13T09:17:25.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-13T09:17:25.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_log_changes_size": "1000", 2024-09-13T09:17:25.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_log_num_shards": "128", 2024-09-13T09:17:25.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-13T09:17:25.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_log_window": "30", 2024-09-13T09:17:25.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-13T09:17:25.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-13T09:17:25.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-13T09:17:25.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-13T09:17:25.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-13T09:17:25.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-13T09:17:25.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-13T09:17:25.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-13T09:17:25.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-13T09:17:25.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-13T09:17:25.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-13T09:17:25.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-13T09:17:25.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dns_name": "", 2024-09-13T09:17:25.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dns_s3website_name": "", 2024-09-13T09:17:25.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dynamic_resharding": "true", 2024-09-13T09:17:25.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-13T09:17:25.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_gc_threads": "true", 2024-09-13T09:17:25.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_lc_threads": "true", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_ops_log": "false", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_quota_threads": "true", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_static_website": "false", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_usage_log": "false", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enforce_swift_acls": "true", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_exit_timeout_secs": "120", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_expose_bucket": "false", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_extended_http_attrs": "", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_filter": "none", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_frontends": "beast port=7480", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-13T09:17:25.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_deferred": "50", 2024-09-13T09:17:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-13T09:17:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_objs": "32", 2024-09-13T09:17:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-13T09:17:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-13T09:17:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-13T09:17:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-13T09:17:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_processor_period": "3600", 2024-09-13T09:17:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-13T09:17:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-13T09:17:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-13T09:17:25.459 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_init_timeout": "300", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_domain": "", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_password": "", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_password_path": "", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_project": "", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_tenant": "", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_token": "", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_token_path": "", 2024-09-13T09:17:25.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_user": "", 2024-09-13T09:17:25.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_api_version": "2", 2024-09-13T09:17:25.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_domain": "", 2024-09-13T09:17:25.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_password": "", 2024-09-13T09:17:25.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_project": "", 2024-09-13T09:17:25.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-13T09:17:25.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_user": "", 2024-09-13T09:17:25.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-13T09:17:25.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-13T09:17:25.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-13T09:17:25.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-13T09:17:25.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-13T09:17:25.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_url": "", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_debug_interval": "-1", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_lock_max_time": "90", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_max_objs": "32", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_max_rules": "1000", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_max_worker": "3", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_thread_delay": "0", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_dnattr": "uid", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_searchfilter": "", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-13T09:17:25.462 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_log_http_headers": "", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_log_object_name_utc": "false", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_attr_name_len": "0", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_attr_size": "0", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_chunk_size": "4194304", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-13T09:17:25.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-13T09:17:25.464 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-13T09:17:25.464 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_listing_results": "1000", 2024-09-13T09:17:25.464 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_notify_retries": "10", 2024-09-13T09:17:25.464 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-13T09:17:25.464 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_put_param_size": "1048576", 2024-09-13T09:17:25.464 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_put_size": "5368709120", 2024-09-13T09:17:25.464 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_slo_entries": "1000", 2024-09-13T09:17:25.464 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_md_log_max_shards": "64", 2024-09-13T09:17:25.464 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-13T09:17:25.464 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-13T09:17:25.464 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-13T09:17:25.464 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-13T09:17:25.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_mp_lock_max_time": "600", 2024-09-13T09:17:25.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-13T09:17:25.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-13T09:17:25.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-13T09:17:25.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-13T09:17:25.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-13T09:17:25.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-13T09:17:25.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-13T09:17:25.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-13T09:17:25.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_max_gc": "300", 2024-09-13T09:17:25.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-13T09:17:25.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_num_async_rados_threads": "32", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_num_control_oids": "8", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_numa_node": "-1", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_objexp_chunk_size": "100", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_objexp_gc_interval": "600", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-13T09:17:25.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_op_thread_timeout": "600", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_op_tracing": "false", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_opa_token": "", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_opa_url": "", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_opa_verify_ssl": "true", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ops_log_rados": "false", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ops_log_socket_path": "", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-13T09:17:25.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_period_push_interval": "2.000000", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_print_continue": "true", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_rados_tracing": "false", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_realm": "", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_realm_id": "", 2024-09-13T09:17:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-13T09:17:25.469 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_region": "", 2024-09-13T09:17:25.469 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-13T09:17:25.469 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-13T09:17:25.469 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-13T09:17:25.469 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-13T09:17:25.469 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_request_uri": "", 2024-09-13T09:17:25.469 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_batch_size": "64", 2024-09-13T09:17:25.469 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-13T09:17:25.469 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_max_aio": "128", 2024-09-13T09:17:25.469 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_num_logs": "16", 2024-09-13T09:17:25.469 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_thread_interval": "600", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_resolve_cname": "false", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_run_sync_thread": "true", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_scheduler_type": "throttler", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_script_uri": "", 2024-09-13T09:17:25.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_service_provider_name": "", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_client_id": "", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_client_secret": "", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_entry": "sts", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_key": "sts", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_min_session_duration": "900", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_token_introspection_url": "", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_account_in_url": "false", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_auth_entry": "auth", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_auth_url": "", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_custom_header": "", 2024-09-13T09:17:25.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_need_stats": "true", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_tenant_name": "", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_token_expiration": "86400", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_url": "", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_url_prefix": "swift", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_lease_period": "120", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-13T09:17:25.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_thread_pool_size": "512", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_comment": "", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_createby": "", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_encoding": "", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_flag": "false", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_origin": "", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-13T09:17:25.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_tracker": "", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_trust_forwarded_https": "false", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_usage_max_shards": "32", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_usage_max_user_shards": "1", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_use_opa_authz": "false", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_max_buckets": "1000", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_policies_max_num": "100", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-13T09:17:25.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_unique_email": "true", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_verify_ssl": "true", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zone": "", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zone_id": "", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zonegroup": "", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zonegroup_id": "", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rgwlc_auto_session_clear": "true", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-13T09:17:25.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_block_size": "4096", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_size": "536870912", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-13T09:17:25.476 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_index_type": "binary_search", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_partition_filters": "false", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_perf": "false", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "run_dir": "/var/run/ceph", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_block_create": "true", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_cache_lru_size": "67108864", 2024-09-13T09:17:25.477 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_default_max_object_size": "16777216", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_device_size": "53687091200", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_journal_batch_capacity": "16", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_main_device_type": "SSD", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-13T09:17:25.478 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-13T09:17:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_segment_size": "67108864", 2024-09-13T09:17:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: "set_keepcaps": "false", 2024-09-13T09:17:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: "setgroup": "", 2024-09-13T09:17:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: "setuser": "", 2024-09-13T09:17:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: "setuser_match_path": "", 2024-09-13T09:17:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-13T09:17:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: "thp": "false", 2024-09-13T09:17:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: "threadpool_default_timeout": "60", 2024-09-13T09:17:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-13T09:17:25.479 INFO:teuthology.orchestra.run.smithi032.stdout: "throttler_perf_counter": "true" 2024-09-13T09:17:25.479 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:25.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-13T09:17:25.626 INFO:teuthology.orchestra.run.smithi032.stdout:mon.a: {} 2024-09-13T09:17:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:17:25.639 INFO:teuthology.orchestra.run.smithi032.stdout:mon.b: {} 2024-09-13T09:17:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:17:25.649 INFO:teuthology.orchestra.run.smithi032.stdout:mon.c: {} 2024-09-13T09:17:25.649 INFO:teuthology.orchestra.run.smithi032.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:17:25.667 DEBUG:teuthology.orchestra.run.smithi046:> type ceph-objectstore-tool 2024-09-13T09:17:25.672 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:17:25.673 DEBUG:teuthology.orchestra.run.smithi032:> type ceph-objectstore-tool 2024-09-13T09:17:25.676 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:17:25.677 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-13T09:17:25.688 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-13T09:17:25.688 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-13T09:17:25.688 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:17:25.688 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:17:25.688 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:17:25.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-13T09:17:25.689 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-13T09:17:25.689 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-13T09:17:25.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:17:25.690 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-13T09:17:25.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:17:25.690 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-13T09:17:25.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:17:25.691 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-13T09:17:25.714 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:25.710+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:25.743 INFO:teuthology.orchestra.run.smithi032.stdout:mapped writes DISABLED 2024-09-13T09:17:25.743 INFO:teuthology.orchestra.run.smithi032.stdout:mapped reads DISABLED 2024-09-13T09:17:25.743 INFO:teuthology.orchestra.run.smithi032.stdout:Seed set to 9123 2024-09-13T09:17:25.791 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:25.788+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:25.847 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:25.847 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:25.847 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:25.847 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:25.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:17:25.891 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:25.892 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:25.890+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:25.923 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:25.924 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:25.955 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:25.956 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:25.974 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:25.974 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:25.992 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:25.988+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:25.995 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:25.995 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:26.004 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:26.005 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:26.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:17:26.015 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:26.016 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:26.043 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-09-13T09:17:26.040+0000 7f1d86e9f640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-13T09:17:26.048 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:26.048 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:26.067 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:26.093 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:26.086+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:26.126 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:26.126 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:26.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:17:26.194 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:26.190+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:26.294 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:26.294 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:26.295 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:26.290+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:26.308 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:17:26.396 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:26.392+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:26.429 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:26.429 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:26.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:17:26.497 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:26.490+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:26.561 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:17:26.562 INFO:teuthology.orchestra.run.smithi032.stderr:marked out osd.0. 2024-09-13T09:17:26.597 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:26.594+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:26.617 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:26.617 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:26.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:17:26.723 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:26.720+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:26.795 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:26.796 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:26.799 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:26.794+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:26.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:17:26.900 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:26.896+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:26.970 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:26.971 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:26.985 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:17:27.026 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:27.024+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:27.101 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:27.096+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:27.153 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:27.154 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:27.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:17:27.210 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:27.208+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:27.304 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:27.302+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:27.337 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:27.337 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:27.352 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:17:27.404 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:27.402+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:27.478 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:27.478 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:27.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:17:27.505 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:27.498+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:27.606 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:27.604+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:27.616 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:27.617 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:27.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:17:27.707 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:27.702+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:27.769 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:27.769 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:27.787 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:17:27.808 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:27.804+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:27.908 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:27.902+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:27.955 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:27.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:17:28.014 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:28.010+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:28.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:17:28.109 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:28.106+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:28.135 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:28.135 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:28.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:17:28.237 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:28.234+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:28.242 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:28.242 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:28.258 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:28.258 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:28.273 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:28.274 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:28.290 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:28.290 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:28.310 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:28.310 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:28.311 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:28.308+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:28.323 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:28.323 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:28.331 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:28.331 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:28.337 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:17:28.350 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:28.371 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:28.434 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:28.432+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:28.509 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:28.509 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:28.509 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:17:28.509 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:17:28.509 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:28.513 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:28.512+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:28.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:17:28.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:17:28.642 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:28.640+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:28.643 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:28.643 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:17:28.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:17:28.643 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:17:28.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.644 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4263.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:17:28.644 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:17.489199+0000", 2024-09-13T09:17:28.644 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.152779176999999, 2024-09-13T09:17:28.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010815508999999999, 2024-09-13T09:17:28.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:28.644 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:28.644 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:28.644 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4263", 2024-09-13T09:17:28.644 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/4165219566", 2024-09-13T09:17:28.644 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:17:28.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.644 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:28.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:28.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.489199+0000", 2024-09-13T09:17:28.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:28.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.489199+0000", 2024-09-13T09:17:28.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:28.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.489191+0000", 2024-09-13T09:17:28.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999914 2024-09-13T09:17:28.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:28.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.489213+0000", 2024-09-13T09:17:28.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2497000000000002e-05 2024-09-13T09:17:28.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:28.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.489248+0000", 2024-09-13T09:17:28.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4972000000000003e-05 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.489271+0000", 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3173e-05 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.491198+0000", 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019266489999999999 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:17:28.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.491235+0000", 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7066000000000001e-05 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.497098+0000", 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058632210000000001 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.497444+0000", 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034528100000000002 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:28.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.498073+0000", 2024-09-13T09:17:28.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00062919199999999999 2024-09-13T09:17:28.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:28.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.499409+0000", 2024-09-13T09:17:28.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00133632 2024-09-13T09:17:28.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:28.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.499823+0000", 2024-09-13T09:17:28.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041357999999999998 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.499907+0000", 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3799000000000002e-05 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.500015+0000", 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108119 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4263.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:17.499643+0000", 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.142335398, 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000382474, 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.499643+0000", 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.499643+0000", 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.499641+0000", 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.499653+0000", 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1556000000000001e-05 2024-09-13T09:17:28.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.499679+0000", 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6247000000000001e-05 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.499699+0000", 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0242999999999999e-05 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.499800+0000", 2024-09-13T09:17:28.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100566 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.499813+0000", 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2792999999999999e-05 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.500025+0000", 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000212862 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4263.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:17:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:17.504363+0000", 2024-09-13T09:17:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.137614927, 2024-09-13T09:17:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00039722299999999999, 2024-09-13T09:17:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:17:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4263", 2024-09-13T09:17:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/4165219566", 2024-09-13T09:17:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.504363+0000", 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.504363+0000", 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.504361+0000", 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.504375+0000", 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.378e-05 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.504407+0000", 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2356e-05 2024-09-13T09:17:28.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.504423+0000", 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5396999999999998e-05 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.504535+0000", 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011233199999999999 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:28.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.504761+0000", 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022553199999999999 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4263.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:17.505319+0000", 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.136659752, 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034702099999999998, 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:17:28.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4263", 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/4165219566", 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 3 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.505319+0000", 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:28.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.505319+0000", 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.505317+0000", 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.505327+0000", 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.044e-05 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.505353+0000", 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5282999999999998e-05 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.505364+0000", 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1293e-05 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.505471+0000", 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107323 2024-09-13T09:17:28.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:17.505666+0000", 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019443499999999999 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4263.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:18.500914+0000", 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.141064635999999, 2024-09-13T09:17:28.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064715969999999999, 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4263", 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/4165219566", 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.500914+0000", 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:28.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.500914+0000", 2024-09-13T09:17:28.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:28.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.500911+0000", 2024-09-13T09:17:28.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:28.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:28.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.500933+0000", 2024-09-13T09:17:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2056999999999999e-05 2024-09-13T09:17:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.500967+0000", 2024-09-13T09:17:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3711999999999998e-05 2024-09-13T09:17:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.500987+0000", 2024-09-13T09:17:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.037e-05 2024-09-13T09:17:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:28.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.501136+0000", 2024-09-13T09:17:28.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014940299999999999 2024-09-13T09:17:28.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:17:28.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.501147+0000", 2024-09-13T09:17:28.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0803e-05 2024-09-13T09:17:28.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:28.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.504660+0000", 2024-09-13T09:17:28.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035127159999999999 2024-09-13T09:17:28.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:28.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.504867+0000", 2024-09-13T09:17:28.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020724799999999999 2024-09-13T09:17:28.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-13T09:17:28.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.505080+0000", 2024-09-13T09:17:28.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021297199999999999 2024-09-13T09:17:28.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:28.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.506516+0000", 2024-09-13T09:17:28.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014355850000000001 2024-09-13T09:17:28.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:28.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.507244+0000", 2024-09-13T09:17:28.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072847899999999998 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.507326+0000", 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2201999999999997e-05 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.507385+0000", 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8797e-05 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4263.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:18.507066+0000", 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.134912687, 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000332937, 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.507066+0000", 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.507066+0000", 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.507064+0000", 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.507075+0000", 2024-09-13T09:17:28.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1029999999999999e-05 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.507100+0000", 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4816000000000001e-05 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.507113+0000", 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.325e-05 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:28.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.507218+0000", 2024-09-13T09:17:28.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105369 2024-09-13T09:17:28.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:28.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.507235+0000", 2024-09-13T09:17:28.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6994e-05 2024-09-13T09:17:28.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:28.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:18.507399+0000", 2024-09-13T09:17:28.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016321799999999999 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4263.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:19.503415+0000", 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1385634319999998, 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076554730000000003, 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:28.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4263", 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/4165219566", 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.503415+0000", 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.503415+0000", 2024-09-13T09:17:28.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.503412+0000", 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.503428+0000", 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5852999999999999e-05 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.503463+0000", 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5429000000000002e-05 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.503480+0000", 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.503632+0000", 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151459 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.503651+0000", 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8995999999999999e-05 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.507759+0000", 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0041077220000000003 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.507991+0000", 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000232308 2024-09-13T09:17:28.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.508424+0000", 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043293299999999998 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.510346+0000", 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001921762 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:28.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.510931+0000", 2024-09-13T09:17:28.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000585185 2024-09-13T09:17:28.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:28.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.511009+0000", 2024-09-13T09:17:28.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8040000000000005e-05 2024-09-13T09:17:28.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:28.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.511070+0000", 2024-09-13T09:17:28.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1501999999999995e-05 2024-09-13T09:17:28.684 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4263.0:7 2.2 e21/18 ondisk, result = 0)", 2024-09-13T09:17:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:19.510757+0000", 2024-09-13T09:17:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1312215129999998, 2024-09-13T09:17:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032556399999999997, 2024-09-13T09:17:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.510757+0000", 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.510757+0000", 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.510755+0000", 2024-09-13T09:17:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.510770+0000", 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5362999999999998e-05 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.510806+0000", 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5726000000000003e-05 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.510832+0000", 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6230000000000001e-05 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.510908+0000", 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5893e-05 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.510922+0000", 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3933e-05 2024-09-13T09:17:28.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:28.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:28.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:28.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:19.511082+0000", 2024-09-13T09:17:28.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016068199999999999 2024-09-13T09:17:28.689 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.689 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:28.689 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.689 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:28.689 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:28.689 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:28.690 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:17:28.715 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:28.712+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:28.812 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:28.812 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:28.812 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:17:28.812 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:28.816 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:28.810+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:28.826 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:17:28.916 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:28.910+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:28.998 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:28.998 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:28.998 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:17:28.998 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:17:28.999 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:29.012 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:17:29.054 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:29.050+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:29.120 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:29.114+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:29.182 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:29.182 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:17:29.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:17:29.182 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [] 2024-09-13T09:17:29.183 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:29.196 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:17:29.238 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:29.234+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:29.322 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:29.318+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:29.363 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:29.364 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:29.364 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:17:29.364 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:29.378 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:17:29.422 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:29.420+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:29.523 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:29.518+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:29.553 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:17:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:17:29.553 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:29.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:17:29.625 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:29.620+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:29.692 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-13T09:17:29.748 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:29.748 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:17:29.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:17:29.748 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:17:29.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.749 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.749 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:29.749 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.350203712999999, 2024-09-13T09:17:29.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077478195999999999, 2024-09-13T09:17:29.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:17:29.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.750 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:29.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:29.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392636+0000", 2024-09-13T09:17:29.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:29.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392651+0000", 2024-09-13T09:17:29.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:17:29.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392674+0000", 2024-09-13T09:17:29.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.264e-05 2024-09-13T09:17:29.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392691+0000", 2024-09-13T09:17:29.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7203000000000001e-05 2024-09-13T09:17:29.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392780+0000", 2024-09-13T09:17:29.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8671999999999994e-05 2024-09-13T09:17:29.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392794+0000", 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.466664+0000", 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073869913999999995 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467139+0000", 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047488599999999999 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467664+0000", 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052530600000000002 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469125+0000", 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001460432 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469847+0000", 2024-09-13T09:17:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072197799999999996 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470006+0000", 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015955900000000001 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470116+0000", 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109836 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531091+0000", 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.211750740999999, 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010986811000000001, 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 34 2024-09-13T09:17:29.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.758 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531091+0000", 2024-09-13T09:17:29.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531091+0000", 2024-09-13T09:17:29.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531090+0000", 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531096+0000", 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.806e-06 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531108+0000", 2024-09-13T09:17:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.224e-05 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531117+0000", 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6030000000000005e-06 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.536352+0000", 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052353260000000002 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.536628+0000", 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027556500000000002 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.536981+0000", 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035341700000000002 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.541700+0000", 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047184100000000001 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.541795+0000", 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5031999999999994e-05 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.541955+0000", 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160056 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542078+0000", 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012296200000000001 2024-09-13T09:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531127+0000", 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.211714815000001, 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011389865000000001, 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 35 2024-09-13T09:17:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531127+0000", 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531127+0000", 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531126+0000", 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531274+0000", 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014800800000000001 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531288+0000", 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3867e-05 2024-09-13T09:17:29.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531297+0000", 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9330000000000007e-06 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.538727+0000", 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.007429639 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.538949+0000", 2024-09-13T09:17:29.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022249499999999999 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.539160+0000", 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000211079 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542252+0000", 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030918460000000001 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542280+0000", 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7433e-05 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542409+0000", 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012945599999999999 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542517+0000", 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107725 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531307+0000", 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.211534339, 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012285895, 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 36 2024-09-13T09:17:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531307+0000", 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531307+0000", 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531306+0000", 2024-09-13T09:17:29.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:29.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531313+0000", 2024-09-13T09:17:29.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3130000000000004e-06 2024-09-13T09:17:29.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531322+0000", 2024-09-13T09:17:29.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.85e-06 2024-09-13T09:17:29.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531329+0000", 2024-09-13T09:17:29.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9600000000000003e-06 2024-09-13T09:17:29.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.540210+0000", 2024-09-13T09:17:29.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088810439999999994 2024-09-13T09:17:29.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.540477+0000", 2024-09-13T09:17:29.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000267438 2024-09-13T09:17:29.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.540862+0000", 2024-09-13T09:17:29.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038524400000000003 2024-09-13T09:17:29.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543383+0000", 2024-09-13T09:17:29.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002520764 2024-09-13T09:17:29.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543447+0000", 2024-09-13T09:17:29.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4063e-05 2024-09-13T09:17:29.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543538+0000", 2024-09-13T09:17:29.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1268999999999996e-05 2024-09-13T09:17:29.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543593+0000", 2024-09-13T09:17:29.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000002e-05 2024-09-13T09:17:29.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.775 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531337+0000", 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.211504828999999, 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013657717, 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.776 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531337+0000", 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531337+0000", 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531336+0000", 2024-09-13T09:17:29.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:29.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531344+0000", 2024-09-13T09:17:29.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5599999999999996e-06 2024-09-13T09:17:29.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531353+0000", 2024-09-13T09:17:29.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8869999999999998e-06 2024-09-13T09:17:29.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531360+0000", 2024-09-13T09:17:29.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9100000000000005e-06 2024-09-13T09:17:29.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542600+0000", 2024-09-13T09:17:29.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011239638999999999 2024-09-13T09:17:29.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542747+0000", 2024-09-13T09:17:29.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014643499999999999 2024-09-13T09:17:29.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542947+0000", 2024-09-13T09:17:29.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000200149 2024-09-13T09:17:29.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543918+0000", 2024-09-13T09:17:29.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00097078599999999996 2024-09-13T09:17:29.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.544760+0000", 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00084245000000000001 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.544898+0000", 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013783900000000001 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.544995+0000", 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6766000000000002e-05 2024-09-13T09:17:29.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.631401+0000", 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.111440332000001, 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010376106, 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 86 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631401+0000", 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631401+0000", 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631400+0000", 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631411+0000", 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1029999999999999e-05 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631429+0000", 2024-09-13T09:17:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7907e-05 2024-09-13T09:17:29.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631443+0000", 2024-09-13T09:17:29.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4283e-05 2024-09-13T09:17:29.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.636777+0000", 2024-09-13T09:17:29.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053336190000000004 2024-09-13T09:17:29.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.636951+0000", 2024-09-13T09:17:29.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174325 2024-09-13T09:17:29.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.637168+0000", 2024-09-13T09:17:29.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021682200000000001 2024-09-13T09:17:29.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641519+0000", 2024-09-13T09:17:29.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0043510759999999997 2024-09-13T09:17:29.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641638+0000", 2024-09-13T09:17:29.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118573 2024-09-13T09:17:29.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641721+0000", 2024-09-13T09:17:29.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2803e-05 2024-09-13T09:17:29.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641777+0000", 2024-09-13T09:17:29.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6771999999999998e-05 2024-09-13T09:17:29.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.789 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.631461+0000", 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.111380859, 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010750297000000001, 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 87 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.790 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631461+0000", 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631461+0000", 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631460+0000", 2024-09-13T09:17:29.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631618+0000", 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015849899999999999 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631641+0000", 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2912999999999999e-05 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631657+0000", 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6305999999999999e-05 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.638673+0000", 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0070156300000000001 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.638802+0000", 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129049 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.639028+0000", 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022574100000000001 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641921+0000", 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002893648 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641947+0000", 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5307e-05 2024-09-13T09:17:29.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642120+0000", 2024-09-13T09:17:29.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017344499999999999 2024-09-13T09:17:29.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642211+0000", 2024-09-13T09:17:29.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0876000000000002e-05 2024-09-13T09:17:29.795 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.795 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.795 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.796 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.796 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.631676+0000", 2024-09-13T09:17:29.796 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.111165423999999, 2024-09-13T09:17:29.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011331733, 2024-09-13T09:17:29.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.796 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.796 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.796 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.796 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.796 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 88 2024-09-13T09:17:29.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.797 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631676+0000", 2024-09-13T09:17:29.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631676+0000", 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631675+0000", 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631687+0000", 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2163e-05 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631705+0000", 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7876999999999998e-05 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631722+0000", 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.685e-05 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.640114+0000", 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083919979999999995 2024-09-13T09:17:29.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.640389+0000", 2024-09-13T09:17:29.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027471800000000002 2024-09-13T09:17:29.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.640788+0000", 2024-09-13T09:17:29.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00039957000000000002 2024-09-13T09:17:29.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642474+0000", 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001686014 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642883+0000", 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040869700000000002 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642957+0000", 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4239000000000005e-05 2024-09-13T09:17:29.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.643008+0000", 2024-09-13T09:17:29.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0917000000000001e-05 2024-09-13T09:17:29.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.802 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.802 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.802 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.797962+0000", 2024-09-13T09:17:29.802 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.944879450999998, 2024-09-13T09:17:29.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01079161, 2024-09-13T09:17:29.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 110 2024-09-13T09:17:29.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.803 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797962+0000", 2024-09-13T09:17:29.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797962+0000", 2024-09-13T09:17:29.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797960+0000", 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797974+0000", 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4545999999999999e-05 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797999+0000", 2024-09-13T09:17:29.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4573e-05 2024-09-13T09:17:29.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798017+0000", 2024-09-13T09:17:29.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7940000000000001e-05 2024-09-13T09:17:29.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.804966+0000", 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069496469999999998 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.805091+0000", 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012414199999999999 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.805276+0000", 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018536799999999999 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808455+0000", 2024-09-13T09:17:29.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00317914 2024-09-13T09:17:29.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808580+0000", 2024-09-13T09:17:29.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012457500000000001 2024-09-13T09:17:29.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808654+0000", 2024-09-13T09:17:29.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4373000000000001e-05 2024-09-13T09:17:29.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808754+0000", 2024-09-13T09:17:29.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9869000000000002e-05 2024-09-13T09:17:29.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.809 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.809 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.809 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798038+0000", 2024-09-13T09:17:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.944803615000001, 2024-09-13T09:17:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012627325999999999, 2024-09-13T09:17:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 111 2024-09-13T09:17:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798038+0000", 2024-09-13T09:17:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798038+0000", 2024-09-13T09:17:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798037+0000", 2024-09-13T09:17:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798048+0000", 2024-09-13T09:17:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1194e-05 2024-09-13T09:17:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798074+0000", 2024-09-13T09:17:29.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5516000000000001e-05 2024-09-13T09:17:29.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798088+0000", 2024-09-13T09:17:29.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4663e-05 2024-09-13T09:17:29.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.806212+0000", 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081233810000000007 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.806328+0000", 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011605200000000001 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.806514+0000", 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018678299999999999 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808565+0000", 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020504439999999998 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810494+0000", 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019292109999999999 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810569+0000", 2024-09-13T09:17:29.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4436000000000004e-05 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810665+0000", 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6842999999999994e-05 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798108+0000", 2024-09-13T09:17:29.816 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.944733988999999, 2024-09-13T09:17:29.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014039507999999999, 2024-09-13T09:17:29.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.817 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112 2024-09-13T09:17:29.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.817 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798108+0000", 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798108+0000", 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798107+0000", 2024-09-13T09:17:29.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798267+0000", 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160116 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798294+0000", 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6923e-05 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798317+0000", 2024-09-13T09:17:29.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3693000000000002e-05 2024-09-13T09:17:29.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.807303+0000", 2024-09-13T09:17:29.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089853680000000005 2024-09-13T09:17:29.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.807428+0000", 2024-09-13T09:17:29.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012510200000000001 2024-09-13T09:17:29.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.807616+0000", 2024-09-13T09:17:29.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188145 2024-09-13T09:17:29.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810479+0000", 2024-09-13T09:17:29.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028628949999999998 2024-09-13T09:17:29.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811969+0000", 2024-09-13T09:17:29.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001489661 2024-09-13T09:17:29.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812047+0000", 2024-09-13T09:17:29.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8296999999999999e-05 2024-09-13T09:17:29.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812147+0000", 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100375 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798339+0000", 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.944502391, 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013996181999999999, 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.823 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798339+0000", 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798339+0000", 2024-09-13T09:17:29.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798338+0000", 2024-09-13T09:17:29.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:29.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798357+0000", 2024-09-13T09:17:29.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.897e-05 2024-09-13T09:17:29.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798404+0000", 2024-09-13T09:17:29.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7119000000000001e-05 2024-09-13T09:17:29.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798421+0000", 2024-09-13T09:17:29.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7187e-05 2024-09-13T09:17:29.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808797+0000", 2024-09-13T09:17:29.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010375880000000001 2024-09-13T09:17:29.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808915+0000", 2024-09-13T09:17:29.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011846200000000001 2024-09-13T09:17:29.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809099+0000", 2024-09-13T09:17:29.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183313 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811848+0000", 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027493439999999999 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812174+0000", 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032562200000000002 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812244+0000", 2024-09-13T09:17:29.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0902000000000007e-05 2024-09-13T09:17:29.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812335+0000", 2024-09-13T09:17:29.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0995999999999994e-05 2024-09-13T09:17:29.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.829 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798442+0000", 2024-09-13T09:17:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.944399631, 2024-09-13T09:17:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015723348000000002, 2024-09-13T09:17:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798442+0000", 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798442+0000", 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798441+0000", 2024-09-13T09:17:29.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:29.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798453+0000", 2024-09-13T09:17:29.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2784e-05 2024-09-13T09:17:29.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798476+0000", 2024-09-13T09:17:29.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2875999999999999e-05 2024-09-13T09:17:29.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798490+0000", 2024-09-13T09:17:29.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3482999999999999e-05 2024-09-13T09:17:29.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809646+0000", 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011155847 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809754+0000", 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108256 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809965+0000", 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000210659 2024-09-13T09:17:29.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811953+0000", 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019889439999999999 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813962+0000", 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020080850000000002 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814040+0000", 2024-09-13T09:17:29.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8184999999999998e-05 2024-09-13T09:17:29.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814165+0000", 2024-09-13T09:17:29.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012563299999999999 2024-09-13T09:17:29.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.836 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.837 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.837 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798506+0000", 2024-09-13T09:17:29.837 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.944335261999999, 2024-09-13T09:17:29.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015834198000000001, 2024-09-13T09:17:29.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 115 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798506+0000", 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798506+0000", 2024-09-13T09:17:29.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798505+0000", 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798658+0000", 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000153005 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798681+0000", 2024-09-13T09:17:29.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2634e-05 2024-09-13T09:17:29.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798697+0000", 2024-09-13T09:17:29.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5696e-05 2024-09-13T09:17:29.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810746+0000", 2024-09-13T09:17:29.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012049331 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810873+0000", 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012754899999999999 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811051+0000", 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000177252 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813948+0000", 2024-09-13T09:17:29.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028975239999999998 2024-09-13T09:17:29.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814188+0000", 2024-09-13T09:17:29.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000239724 2024-09-13T09:17:29.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814255+0000", 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7372999999999993e-05 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814341+0000", 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5270000000000002e-05 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798716+0000", 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.944125416999999, 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015827386999999998, 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798716+0000", 2024-09-13T09:17:29.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798716+0000", 2024-09-13T09:17:29.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798715+0000", 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798727+0000", 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1800000000000001e-05 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798746+0000", 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8992999999999999e-05 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798765+0000", 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9049e-05 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812383+0000", 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013618531999999999 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812498+0000", 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115056 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812682+0000", 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018366899999999999 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814367+0000", 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00168501 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814421+0000", 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4199000000000001e-05 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814492+0000", 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0502999999999999e-05 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814544+0000", 2024-09-13T09:17:29.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2039000000000001e-05 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798781+0000", 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.944060531000002, 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016163997999999999, 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 117 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798781+0000", 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798781+0000", 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798780+0000", 2024-09-13T09:17:29.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:29.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798795+0000", 2024-09-13T09:17:29.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4574e-05 2024-09-13T09:17:29.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798813+0000", 2024-09-13T09:17:29.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8536e-05 2024-09-13T09:17:29.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798829+0000", 2024-09-13T09:17:29.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5866999999999999e-05 2024-09-13T09:17:29.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813236+0000", 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014407229000000001 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813350+0000", 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011376500000000001 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813530+0000", 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179619 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814654+0000", 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011244339999999999 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814768+0000", 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011397599999999999 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814845+0000", 2024-09-13T09:17:29.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7299999999999995e-05 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814945+0000", 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9875000000000002e-05 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:12.288452+0000", 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.454389846000002, 2024-09-13T09:17:29.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010418200000000001, 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288452+0000", 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288452+0000", 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288451+0000", 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288508+0000", 2024-09-13T09:17:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6706999999999998e-05 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288517+0000", 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8339999999999994e-06 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288523+0000", 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5419999999999996e-06 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.291720+0000", 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031973499999999998 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.291765+0000", 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.443e-05 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.291834+0000", 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9053000000000004e-05 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.293879+0000", 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020453619999999998 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.298804+0000", 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049248069999999998 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.298831+0000", 2024-09-13T09:17:29.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7436e-05 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.298870+0000", 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8549e-05 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:12.288530+0000", 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.454311889, 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011049003999999999, 2024-09-13T09:17:29.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 141 2024-09-13T09:17:29.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.863 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288530+0000", 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288530+0000", 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288529+0000", 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288532+0000", 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288536+0000", 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8480000000000003e-06 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288539+0000", 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0479999999999999e-06 2024-09-13T09:17:29.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.292188+0000", 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036491639999999999 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.292357+0000", 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016818200000000001 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.292492+0000", 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013562099999999999 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.293885+0000", 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001393044 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.299519+0000", 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056339670000000001 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.299547+0000", 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7928000000000001e-05 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.299579+0000", 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1547999999999997e-05 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.868 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 16.702741531000001, 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010963133, 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 158 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040255+0000", 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015546599999999999 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040268+0000", 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2860000000000001e-05 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040276+0000", 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5399999999999996e-06 2024-09-13T09:17:29.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.047322+0000", 2024-09-13T09:17:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0070454380000000002 2024-09-13T09:17:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.047443+0000", 2024-09-13T09:17:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012126 2024-09-13T09:17:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.047631+0000", 2024-09-13T09:17:29.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000187558 2024-09-13T09:17:29.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050775+0000", 2024-09-13T09:17:29.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031440000000000001 2024-09-13T09:17:29.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050797+0000", 2024-09-13T09:17:29.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2730000000000001e-05 2024-09-13T09:17:29.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050928+0000", 2024-09-13T09:17:29.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013016500000000001 2024-09-13T09:17:29.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051063+0000", 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013571300000000001 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:13.040286+0000", 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 16.702555368999999, 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011356279, 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:29.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:29.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 159 2024-09-13T09:17:29.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.876 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:29.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:29.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040286+0000", 2024-09-13T09:17:29.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:29.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040286+0000", 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040285+0000", 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040296+0000", 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0473e-05 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:29.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040305+0000", 2024-09-13T09:17:29.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0070000000000003e-06 2024-09-13T09:17:29.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:29.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040315+0000", 2024-09-13T09:17:29.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8500000000000006e-06 2024-09-13T09:17:29.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:29.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.049739+0000", 2024-09-13T09:17:29.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094246899999999995 2024-09-13T09:17:29.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:29.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.049892+0000", 2024-09-13T09:17:29.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015284900000000001 2024-09-13T09:17:29.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:29.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050097+0000", 2024-09-13T09:17:29.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020424500000000001 2024-09-13T09:17:29.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:29.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051387+0000", 2024-09-13T09:17:29.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012901970000000001 2024-09-13T09:17:29.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:29.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051511+0000", 2024-09-13T09:17:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124039 2024-09-13T09:17:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051589+0000", 2024-09-13T09:17:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.873e-05 2024-09-13T09:17:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:29.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051643+0000", 2024-09-13T09:17:29.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3102000000000001e-05 2024-09-13T09:17:29.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.882 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:29.882 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:29.882 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:29.883 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:29.756+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:29.884 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:29.882+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:29.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:17:29.985 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:29.980+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:30.007 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:30.007 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:30.007 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:30.007 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:30.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:17:30.106 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:30.104+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:30.187 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:30.184+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:30.189 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:30.189 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:30.189 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:30.189 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:30.189 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:30.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:17:30.288 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:30.282+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:30.360 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.360 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-13T09:17:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:11.579976+0000", 2024-09-13T09:17:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.779277023999999, 2024-09-13T09:17:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002347348, 2024-09-13T09:17:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.579976+0000", 2024-09-13T09:17:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.579976+0000", 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.579974+0000", 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.579994+0000", 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0083000000000001e-05 2024-09-13T09:17:30.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.580032+0000", 2024-09-13T09:17:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7741999999999999e-05 2024-09-13T09:17:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.580069+0000", 2024-09-13T09:17:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7177e-05 2024-09-13T09:17:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.580185+0000", 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011610199999999999 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.580223+0000", 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7540000000000003e-05 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.582239+0000", 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020165679999999998 2024-09-13T09:17:30.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.582324+0000", 2024-09-13T09:17:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4369000000000005e-05 2024-09-13T09:17:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:57 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-13T09:17:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:11.586713+0000", 2024-09-13T09:17:30.365 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.772539837, 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0023464390000000001, 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586713+0000", 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586713+0000", 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586712+0000", 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586767+0000", 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5726999999999997e-05 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586797+0000", 2024-09-13T09:17:30.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9266999999999999e-05 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586811+0000", 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4679e-05 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586916+0000", 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010517 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586952+0000", 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5228999999999998e-05 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.588980+0000", 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020282009999999999 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.589060+0000", 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0023000000000003e-05 2024-09-13T09:17:30.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:11.608699+0000", 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.750554352999998, 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025443969999999999, 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608699+0000", 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608699+0000", 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608697+0000", 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608757+0000", 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9552999999999997e-05 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608793+0000", 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5902999999999998e-05 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608812+0000", 2024-09-13T09:17:30.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9233e-05 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608987+0000", 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017490600000000001 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.609022+0000", 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4946000000000001e-05 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.611165+0000", 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021434539999999999 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.611243+0000", 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8242000000000001e-05 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:129 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'87, mlcod=16'85)", 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.288944+0000", 2024-09-13T09:17:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.070309515999998, 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0048963449999999999, 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288944+0000", 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288944+0000", 2024-09-13T09:17:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288942+0000", 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288960+0000", 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8077e-05 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288989+0000", 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9476e-05 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289003+0000", 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3046e-05 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289103+0000", 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010054 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289137+0000", 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3575999999999998e-05 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293750+0000", 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046137610000000001 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293840+0000", 2024-09-13T09:17:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9698999999999996e-05 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:130 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'88, mlcod=16'85)", 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.289290+0000", 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.069963412, 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046447700000000003, 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289290+0000", 2024-09-13T09:17:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289290+0000", 2024-09-13T09:17:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289289+0000", 2024-09-13T09:17:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289299+0000", 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6430000000000007e-06 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289315+0000", 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.609e-05 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289324+0000", 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3430000000000002e-06 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289718+0000", 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039453699999999998 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289792+0000", 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3786000000000001e-05 2024-09-13T09:17:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293848+0000", 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0040556020000000002 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293935+0000", 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6749000000000006e-05 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:131 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.289777+0000", 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.069476562999998, 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005922531, 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289777+0000", 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289777+0000", 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289775+0000", 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289961+0000", 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018587200000000001 2024-09-13T09:17:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289993+0000", 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2007e-05 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290016+0000", 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3025999999999999e-05 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290280+0000", 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026384800000000001 2024-09-13T09:17:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290318+0000", 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7945999999999999e-05 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295642+0000", 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0053246159999999999 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295699+0000", 2024-09-13T09:17:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6898999999999999e-05 2024-09-13T09:17:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:132 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-13T09:17:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.290371+0000", 2024-09-13T09:17:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.068882114000001, 2024-09-13T09:17:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0053750650000000001, 2024-09-13T09:17:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290371+0000", 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290371+0000", 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290369+0000", 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290384+0000", 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5133999999999999e-05 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290419+0000", 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4555999999999999e-05 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290437+0000", 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8343e-05 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291376+0000", 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00093908299999999995 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291410+0000", 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3883000000000001e-05 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295705+0000", 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042944790000000004 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295746+0000", 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1496999999999999e-05 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:133 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.290748+0000", 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.068505092999999, 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0050401270000000001, 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290748+0000", 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290748+0000", 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290747+0000", 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290761+0000", 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4236000000000001e-05 2024-09-13T09:17:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290788+0000", 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6826999999999999e-05 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290804+0000", 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5798999999999999e-05 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291981+0000", 2024-09-13T09:17:30.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011771080000000001 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292015+0000", 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4693000000000001e-05 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295750+0000", 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037345780000000001 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295788+0000", 2024-09-13T09:17:30.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:134 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.291161+0000", 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.068092366999998, 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064130539999999996, 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291161+0000", 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291161+0000", 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291159+0000", 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:30.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291257+0000", 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7498999999999996e-05 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291284+0000", 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291305+0000", 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1773e-05 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292753+0000", 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014473719999999999 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292786+0000", 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3370000000000001e-05 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297518+0000", 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0047318569999999999 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297574+0000", 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6026000000000002e-05 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:135 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'85)", 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.291641+0000", 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.067612267000001, 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0059934710000000002, 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291641+0000", 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291641+0000", 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291639+0000", 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291653+0000", 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3977e-05 2024-09-13T09:17:30.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291684+0000", 2024-09-13T09:17:30.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0298999999999999e-05 2024-09-13T09:17:30.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291701+0000", 2024-09-13T09:17:30.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7847e-05 2024-09-13T09:17:30.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294015+0000", 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002313402 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294047+0000", 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2063000000000001e-05 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297582+0000", 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035351430000000001 2024-09-13T09:17:30.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297634+0000", 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2460000000000003e-05 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'85)", 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.292054+0000", 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.067198893, 2024-09-13T09:17:30.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0073146829999999998, 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292054+0000", 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292054+0000", 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292053+0000", 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292067+0000", 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.411e-05 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292092+0000", 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5173000000000001e-05 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292111+0000", 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8963000000000001e-05 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294704+0000", 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025933800000000002 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294735+0000", 2024-09-13T09:17:30.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0366e-05 2024-09-13T09:17:30.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299316+0000", 2024-09-13T09:17:30.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004581412 2024-09-13T09:17:30.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299369+0000", 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3016000000000002e-05 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:137 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'85)", 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.292515+0000", 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.066738684000001, 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068905700000000004, 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.411 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292515+0000", 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292515+0000", 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292513+0000", 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292586+0000", 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3139999999999994e-05 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292618+0000", 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1878999999999998e-05 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292639+0000", 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0752999999999999e-05 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295880+0000", 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0032411889999999998 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295927+0000", 2024-09-13T09:17:30.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7175999999999997e-05 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299375+0000", 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0034476400000000001 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299405+0000", 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0642999999999998e-05 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:138 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'85)", 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.292944+0000", 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.066309050000001, 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064997630000000004, 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292944+0000", 2024-09-13T09:17:30.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292944+0000", 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292943+0000", 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292957+0000", 2024-09-13T09:17:30.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4610000000000001e-05 2024-09-13T09:17:30.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292982+0000", 2024-09-13T09:17:30.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5003000000000001e-05 2024-09-13T09:17:30.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293000+0000", 2024-09-13T09:17:30.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8142999999999999e-05 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.296939+0000", 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0039383700000000001 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.296972+0000", 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2716999999999999e-05 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299412+0000", 2024-09-13T09:17:30.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024409340000000001 2024-09-13T09:17:30.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299444+0000", 2024-09-13T09:17:30.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.154e-05 2024-09-13T09:17:30.420 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.420 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.420 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:139 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'85)", 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.293238+0000", 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.066015166, 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062557749999999999, 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293238+0000", 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293238+0000", 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293237+0000", 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293250+0000", 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.379e-05 2024-09-13T09:17:30.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293276+0000", 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6126e-05 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293295+0000", 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8904000000000001e-05 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297711+0000", 2024-09-13T09:17:30.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044153789999999997 2024-09-13T09:17:30.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297745+0000", 2024-09-13T09:17:30.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4499999999999998e-05 2024-09-13T09:17:30.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299450+0000", 2024-09-13T09:17:30.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00170436 2024-09-13T09:17:30.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299494+0000", 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4302999999999999e-05 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'85)", 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.293606+0000", 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.065646889, 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066098429999999998, 2024-09-13T09:17:30.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.426 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.426 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293606+0000", 2024-09-13T09:17:30.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293606+0000", 2024-09-13T09:17:30.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293605+0000", 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293707+0000", 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010204600000000001 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293732+0000", 2024-09-13T09:17:30.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5429999999999999e-05 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293749+0000", 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6390000000000001e-05 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.298325+0000", 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045769440000000003 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.298370+0000", 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4187000000000002e-05 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300157+0000", 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017874690000000001 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300216+0000", 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9067000000000002e-05 2024-09-13T09:17:30.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:141 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'85)", 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.294217+0000", 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.065036654, 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067059559999999999, 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294217+0000", 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294217+0000", 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294215+0000", 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:30.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294229+0000", 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3974e-05 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294261+0000", 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1903000000000003e-05 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294280+0000", 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9589000000000002e-05 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299531+0000", 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0052504029999999998 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299567+0000", 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6427000000000002e-05 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300868+0000", 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013010470000000001 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300923+0000", 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4203000000000003e-05 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:142 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'100, mlcod=16'98)", 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.304015+0000", 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.055238255999999, 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024443709999999999, 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304015+0000", 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304015+0000", 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304013+0000", 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304261+0000", 2024-09-13T09:17:30.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024744099999999999 2024-09-13T09:17:30.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304284+0000", 2024-09-13T09:17:30.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3370000000000002e-05 2024-09-13T09:17:30.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304298+0000", 2024-09-13T09:17:30.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3502999999999999e-05 2024-09-13T09:17:30.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304384+0000", 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6650000000000006e-05 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304424+0000", 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9941999999999998e-05 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.306398+0000", 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019733390000000002 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.306459+0000", 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1869e-05 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:143 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.304318+0000", 2024-09-13T09:17:30.439 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.054935608000001, 2024-09-13T09:17:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029677929999999998, 2024-09-13T09:17:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304318+0000", 2024-09-13T09:17:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304318+0000", 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304317+0000", 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304426+0000", 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109209 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304450+0000", 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4090000000000001e-05 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304462+0000", 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2480000000000001e-05 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.305073+0000", 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00061091499999999996 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.305112+0000", 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8523999999999999e-05 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307197+0000", 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020850439999999999 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307285+0000", 2024-09-13T09:17:30.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8684999999999995e-05 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:144 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.304477+0000", 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.054776328999999, 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029682900000000002, 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304477+0000", 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304477+0000", 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304476+0000", 2024-09-13T09:17:30.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304603+0000", 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012634500000000001 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304620+0000", 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7450000000000001e-05 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304629+0000", 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4599999999999992e-06 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.305775+0000", 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001145725 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.305808+0000", 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2932999999999998e-05 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307394+0000", 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015858269999999999 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307445+0000", 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1329999999999998e-05 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:145 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.304643+0000", 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.054610670999999, 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0032891209999999999, 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304643+0000", 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304643+0000", 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304642+0000", 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304743+0000", 2024-09-13T09:17:30.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101049 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304775+0000", 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1973e-05 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304783+0000", 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.306544+0000", 2024-09-13T09:17:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017610659999999999 2024-09-13T09:17:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.306577+0000", 2024-09-13T09:17:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3149999999999999e-05 2024-09-13T09:17:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307851+0000", 2024-09-13T09:17:30.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001273843 2024-09-13T09:17:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307932+0000", 2024-09-13T09:17:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0465999999999998e-05 2024-09-13T09:17:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:30.453 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:30.453 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:30.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:17:30.455 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:17:30.456 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:30.452+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:30.555 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:30.552+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:30.617 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:30.617 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:30.617 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:30.617 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:30.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:17:30.635 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:30.636 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:30.652 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:30.653 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:30.656 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:30.654+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:30.670 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:30.670 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:30.686 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:30.708 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:30.728 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:30.748 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:30.753 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:30.753 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:30.757 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:30.750+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:30.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:17:30.769 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:30.857 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:30.852+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:30.883 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:30.884 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:30.888 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:30.888 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0x143c097 2024-09-13T09:17:30.889 INFO:teuthology.orchestra.run.smithi032.stdout:2 trunc from 0x0 to 0x143c097 2024-09-13T09:17:30.898 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:17:30.928 INFO:teuthology.orchestra.run.smithi032.stdout:3 writesame 0xd291ca3 thru 0xd2998e8 data_size 0x1(0x7c46 bytes) 2024-09-13T09:17:30.928 INFO:teuthology.orchestra.run.smithi032.stdout:4 read 0xbf266b9 thru 0xbf2dcda (0x7622 bytes) 2024-09-13T09:17:30.928 INFO:teuthology.orchestra.run.smithi032.stdout:5 write 0x7a9d623 thru 0x7aab12b (0xdb09 bytes) 2024-09-13T09:17:30.935 INFO:teuthology.orchestra.run.smithi032.stdout:6 write 0x441f815 thru 0x4426a07 (0x71f3 bytes) 2024-09-13T09:17:30.942 INFO:teuthology.orchestra.run.smithi032.stdout:7 writesame 0x6ee52bc thru 0x6eeae25 data_size 0x1(0x5b6a bytes) 2024-09-13T09:17:30.958 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:30.956+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:30.976 INFO:teuthology.orchestra.run.smithi032.stdout:10 read 0x8e57070 thru 0x8e5b3b8 (0x4349 bytes) 2024-09-13T09:17:30.976 INFO:teuthology.orchestra.run.smithi032.stdout:12 write 0xcaecc3f thru 0xcaf0076 (0x3438 bytes) 2024-09-13T09:17:30.984 INFO:teuthology.orchestra.run.smithi032.stdout:13 read 0xcc66709 thru 0xcc6be01 (0x56f9 bytes) 2024-09-13T09:17:30.984 INFO:teuthology.orchestra.run.smithi032.stdout:14 punch from 0x2abc2dd to 0x2ac8b1f, (0xc842 bytes) 2024-09-13T09:17:30.984 INFO:teuthology.orchestra.run.smithi032.stdout:15 read 0x2150215 thru 0x2157bb9 (0x79a5 bytes) 2024-09-13T09:17:30.984 INFO:teuthology.orchestra.run.smithi032.stdout:16 write 0x784353c thru 0x7848667 (0x512c bytes) 2024-09-13T09:17:31.000 INFO:teuthology.orchestra.run.smithi032.stdout:17 write 0xd8590cd thru 0xd867ff2 (0xef26 bytes) 2024-09-13T09:17:31.008 INFO:teuthology.orchestra.run.smithi032.stdout:19 read 0x2b07e58 thru 0x2b0a872 (0x2a1b bytes) 2024-09-13T09:17:31.008 INFO:teuthology.orchestra.run.smithi032.stdout:20 read 0xadfa0f9 thru 0xadfdb8f (0x3a97 bytes) 2024-09-13T09:17:31.008 INFO:teuthology.orchestra.run.smithi032.stdout:21 write 0x912f28c thru 0x9134ad5 (0x584a bytes) 2024-09-13T09:17:31.016 INFO:teuthology.orchestra.run.smithi032.stdout:22 write 0xc24f07d thru 0xc25dd82 (0xed06 bytes) 2024-09-13T09:17:31.023 INFO:teuthology.orchestra.run.smithi032.stdout:23 writesame 0xc4670aa thru 0xc475a59 data_size 0x8(0xe9b0 bytes) 2024-09-13T09:17:31.037 INFO:teuthology.orchestra.run.smithi032.stdout:25 punch from 0x8eaaa5 to 0x8f21e3, (0x773e bytes) 2024-09-13T09:17:31.037 INFO:teuthology.orchestra.run.smithi032.stdout:26 write 0x8bb5d1 thru 0x8c9a8e (0xe4be bytes) 2024-09-13T09:17:31.042 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:31.042 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:31.045 INFO:teuthology.orchestra.run.smithi032.stdout:28 write 0x24c0244 thru 0x24cb17d (0xaf3a bytes) 2024-09-13T09:17:31.054 INFO:teuthology.orchestra.run.smithi032.stdout:29 write 0x5874948 thru 0x5878b2d (0x41e6 bytes) 2024-09-13T09:17:31.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:17:31.060 INFO:teuthology.orchestra.run.smithi032.stdout:30 writesame 0x76961c4 thru 0x769b6ef data_size 0x17(0x552c bytes) 2024-09-13T09:17:31.061 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:31.056+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:31.068 INFO:teuthology.orchestra.run.smithi032.stdout:32 read 0x7ed193f thru 0x7ed5b0e (0x41d0 bytes) 2024-09-13T09:17:31.068 INFO:teuthology.orchestra.run.smithi032.stdout:33 read 0x5081a4d thru 0x5083c11 (0x21c5 bytes) 2024-09-13T09:17:31.069 INFO:teuthology.orchestra.run.smithi032.stdout:34 read 0x9feaefb thru 0x9ff213c (0x7242 bytes) 2024-09-13T09:17:31.069 INFO:teuthology.orchestra.run.smithi032.stdout:35 writesame 0x9e80e95 thru 0x9e84b55 data_size 0x1(0x3cc1 bytes) 2024-09-13T09:17:31.091 INFO:teuthology.orchestra.run.smithi032.stdout:36 writesame 0x75231a9 thru 0x7524a91 data_size 0x1(0x18e9 bytes) 2024-09-13T09:17:31.099 INFO:teuthology.orchestra.run.smithi032.stdout:39 writesame 0xc8daa00 thru 0xc8db2fa data_size 0xb(0x8fb bytes) 2024-09-13T09:17:31.101 INFO:teuthology.orchestra.run.smithi032.stdout:40 write 0xb6b2c4d thru 0xb6b7ab5 (0x4e69 bytes) 2024-09-13T09:17:31.107 INFO:teuthology.orchestra.run.smithi032.stdout:41 read 0x5b6454b thru 0x5b7194a (0xd400 bytes) 2024-09-13T09:17:31.108 INFO:teuthology.orchestra.run.smithi032.stdout:43 read 0x93a945b thru 0x93af521 (0x60c7 bytes) 2024-09-13T09:17:31.109 INFO:teuthology.orchestra.run.smithi032.stdout:45 write 0x1766d6c thru 0x1767953 (0xbe8 bytes) 2024-09-13T09:17:31.114 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.114 INFO:teuthology.orchestra.run.smithi032.stdout:47 write 0x8095414 thru 0x80a0356 (0xaf43 bytes) 2024-09-13T09:17:31.121 INFO:teuthology.orchestra.run.smithi032.stdout:48 read 0xd43b937 thru 0xd447e36 (0xc500 bytes) 2024-09-13T09:17:31.121 INFO:teuthology.orchestra.run.smithi032.stdout:50 read 0x3aba781 thru 0x3ac37c7 (0x9047 bytes) 2024-09-13T09:17:31.133 INFO:teuthology.orchestra.run.smithi032.stdout:51 write 0xe106d0e thru 0xe10d2a4 (0x6597 bytes) 2024-09-13T09:17:31.139 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xbe94cfa 2024-09-13T09:17:31.143 INFO:teuthology.orchestra.run.smithi032.stdout:52 trunc from 0xe10d2a5 to 0xbe94cfa 2024-09-13T09:17:31.161 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:31.156+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:31.167 INFO:teuthology.orchestra.run.smithi032.stdout:53 write 0x3d7684f thru 0x3d86826 (0xffd8 bytes) 2024-09-13T09:17:31.174 INFO:teuthology.orchestra.run.smithi032.stdout:55 read 0x2c7956e thru 0x2c7ff45 (0x69d8 bytes) 2024-09-13T09:17:31.174 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.174 INFO:teuthology.orchestra.run.smithi032.stdout:57 punch from 0x7d7cedc to 0x7d8aff0, (0xe114 bytes) 2024-09-13T09:17:31.174 INFO:teuthology.orchestra.run.smithi032.stdout:58 write 0xc799af thru 0xc7ac32 (0x1284 bytes) 2024-09-13T09:17:31.179 INFO:teuthology.orchestra.run.smithi032.stdout:59 punch from 0x4ccd30d to 0x4cd5111, (0x7e04 bytes) 2024-09-13T09:17:31.179 INFO:teuthology.orchestra.run.smithi032.stdout:60 write 0x217bc22 thru 0x2182169 (0x6548 bytes) 2024-09-13T09:17:31.185 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:31.185 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:31.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:17:31.204 INFO:teuthology.orchestra.run.smithi032.stdout:62 write 0xd360e1b thru 0xd36ec0a (0xddf0 bytes) 2024-09-13T09:17:31.212 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.212 INFO:teuthology.orchestra.run.smithi032.stdout:65 read 0xc49ad8a thru 0xc4a242f (0x76a6 bytes) 2024-09-13T09:17:31.212 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.212 INFO:teuthology.orchestra.run.smithi032.stdout:69 write 0xbe6356a thru 0xbe73266 (0xfcfd bytes) 2024-09-13T09:17:31.219 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.219 INFO:teuthology.orchestra.run.smithi032.stdout:71 write 0x227e553 thru 0x228a0e2 (0xbb90 bytes) 2024-09-13T09:17:31.222 INFO:teuthology.orchestra.run.smithi032.stdout:72 write 0x50047ee thru 0x50086ab (0x3ebe bytes) 2024-09-13T09:17:31.229 INFO:teuthology.orchestra.run.smithi032.stdout:73 read 0x3938ee0 thru 0x393d99e (0x4abf bytes) 2024-09-13T09:17:31.246 INFO:teuthology.orchestra.run.smithi032.stdout:74 write 0xed66278 thru 0xed721ac (0xbf35 bytes) 2024-09-13T09:17:31.253 INFO:teuthology.orchestra.run.smithi032.stdout:76 punch from 0x32cf9df to 0x32d8dc5, (0x93e6 bytes) 2024-09-13T09:17:31.254 INFO:teuthology.orchestra.run.smithi032.stdout:79 write 0xbfc5c42 thru 0xbfce8d1 (0x8c90 bytes) 2024-09-13T09:17:31.257 INFO:teuthology.orchestra.run.smithi032.stdout:80 read 0x353725f thru 0x353e330 (0x70d2 bytes) 2024-09-13T09:17:31.257 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.257 INFO:teuthology.orchestra.run.smithi032.stdout:82 write 0x94e5b63 thru 0x94f33db (0xd879 bytes) 2024-09-13T09:17:31.262 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:31.260+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:31.263 INFO:teuthology.orchestra.run.smithi032.stdout:83 write 0x1daf745 thru 0x1db23d7 (0x2c93 bytes) 2024-09-13T09:17:31.270 INFO:teuthology.orchestra.run.smithi032.stdout:84 writesame 0x1da276c thru 0x1dabc8a data_size 0x19(0x951f bytes) 2024-09-13T09:17:31.274 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.274 INFO:teuthology.orchestra.run.smithi032.stdout:89 write 0xb4e768 thru 0xb59161 (0xa9fa bytes) 2024-09-13T09:17:31.277 INFO:teuthology.orchestra.run.smithi032.stdout:90 read 0xc54f760 thru 0xc55f01b (0xf8bc bytes) 2024-09-13T09:17:31.277 INFO:teuthology.orchestra.run.smithi032.stdout:93 read 0xad71a5f thru 0xad77641 (0x5be3 bytes) 2024-09-13T09:17:31.278 INFO:teuthology.orchestra.run.smithi032.stdout:95 read 0x790356d thru 0x7904442 (0xed6 bytes) 2024-09-13T09:17:31.278 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.278 INFO:teuthology.orchestra.run.smithi032.stdout:98 write 0x40296da thru 0x402ab14 (0x143b bytes) 2024-09-13T09:17:31.293 INFO:teuthology.orchestra.run.smithi032.stdout:99 trunc from 0xed721ad to 0x88f3749 2024-09-13T09:17:31.330 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:31.331 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:31.337 INFO:teuthology.orchestra.run.smithi032.stdout:101 write 0xae5e150 thru 0xae667ba (0x866b bytes) 2024-09-13T09:17:31.344 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:17:31.345 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.345 INFO:teuthology.orchestra.run.smithi032.stdout:103 writesame 0x434a882 thru 0x4358fce data_size 0xb(0xe74d bytes) 2024-09-13T09:17:31.353 INFO:teuthology.orchestra.run.smithi032.stdout:104 writesame 0x3f2eb67 thru 0x3f309d2 data_size 0x3b(0x1e6c bytes) 2024-09-13T09:17:31.356 INFO:teuthology.orchestra.run.smithi032.stdout:105 write 0x2c73afe thru 0x2c81124 (0xd627 bytes) 2024-09-13T09:17:31.362 INFO:teuthology.orchestra.run.smithi032.stdout:106 write 0x9cd1352 thru 0x9cd5545 (0x41f4 bytes) 2024-09-13T09:17:31.362 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:31.360+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:31.368 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.369 INFO:teuthology.orchestra.run.smithi032.stdout:108 punch from 0x500f87 to 0x509d80, (0x8df9 bytes) 2024-09-13T09:17:31.369 INFO:teuthology.orchestra.run.smithi032.stdout:109 write 0x89d1f28 thru 0x89d5d2c (0x3e05 bytes) 2024-09-13T09:17:31.375 INFO:teuthology.orchestra.run.smithi032.stdout:110 write 0x63ab11d thru 0x63b5a62 (0xa946 bytes) 2024-09-13T09:17:31.392 INFO:teuthology.orchestra.run.smithi032.stdout:111 write 0xbcbf30c thru 0xbccc17a (0xce6f bytes) 2024-09-13T09:17:31.399 INFO:teuthology.orchestra.run.smithi032.stdout:112 read 0x4912a3d thru 0x491adf9 (0x83bd bytes) 2024-09-13T09:17:31.414 INFO:teuthology.orchestra.run.smithi032.stdout:113 write 0xd61c8bd thru 0xd62642f (0x9b73 bytes) 2024-09-13T09:17:31.421 INFO:teuthology.orchestra.run.smithi032.stdout:114 read 0x252648 thru 0x25df1b (0xb8d4 bytes) 2024-09-13T09:17:31.421 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.421 INFO:teuthology.orchestra.run.smithi032.stdout:116 punch from 0x43d702b to 0x43e08c1, (0x9896 bytes) 2024-09-13T09:17:31.421 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.421 INFO:teuthology.orchestra.run.smithi032.stdout:118 punch from 0x133ff95 to 0x1340d47, (0xdb2 bytes) 2024-09-13T09:17:31.421 INFO:teuthology.orchestra.run.smithi032.stdout:119 write 0x9698d9c thru 0x96a761f (0xe884 bytes) 2024-09-13T09:17:31.428 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.428 INFO:teuthology.orchestra.run.smithi032.stdout:121 write 0x3460fd0 thru 0x346541f (0x4450 bytes) 2024-09-13T09:17:31.434 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.434 INFO:teuthology.orchestra.run.smithi032.stdout:124 punch from 0x89cb7f4 to 0x89d9dc3, (0xe5cf bytes) 2024-09-13T09:17:31.434 INFO:teuthology.orchestra.run.smithi032.stdout:126 punch from 0x9f2d690 to 0x9f3210a, (0x4a7a bytes) 2024-09-13T09:17:31.434 INFO:teuthology.orchestra.run.smithi032.stdout:127 read 0x826de5 thru 0x8361b9 (0xf3d5 bytes) 2024-09-13T09:17:31.435 INFO:teuthology.orchestra.run.smithi032.stdout:129 read 0x898e083 thru 0x899b34e (0xd2cc bytes) 2024-09-13T09:17:31.435 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.450 INFO:teuthology.orchestra.run.smithi032.stdout:132 write 0xe1ee774 thru 0xe1f309e (0x492b bytes) 2024-09-13T09:17:31.456 INFO:teuthology.orchestra.run.smithi032.stdout:133 read 0xa6fcd27 thru 0xa709083 (0xc35d bytes) 2024-09-13T09:17:31.457 INFO:teuthology.orchestra.run.smithi032.stdout:134 punch from 0xf0a90d to 0xf19335, (0xea28 bytes) 2024-09-13T09:17:31.457 INFO:teuthology.orchestra.run.smithi032.stdout:135 write 0x6a7ba16 thru 0x6a83cb4 (0x829f bytes) 2024-09-13T09:17:31.463 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:31.458+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:31.467 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:31.467 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:31.478 INFO:teuthology.orchestra.run.smithi032.stdout:137 trunc from 0xe1f309f to 0x66e2446 2024-09-13T09:17:31.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:17:31.507 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.507 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.507 INFO:teuthology.orchestra.run.smithi032.stdout:142 read 0x574c05f thru 0x574f5d6 (0x3578 bytes) 2024-09-13T09:17:31.521 INFO:teuthology.orchestra.run.smithi032.stdout:143 writesame 0x7355e2b thru 0x735696c data_size 0xb(0xb42 bytes) 2024-09-13T09:17:31.521 INFO:teuthology.orchestra.run.smithi032.stdout:144 punch from 0x5c958b1 to 0x5c9c60a, (0x6d59 bytes) 2024-09-13T09:17:31.521 INFO:teuthology.orchestra.run.smithi032.stdout:145 write 0x548840e thru 0x548d64f (0x5242 bytes) 2024-09-13T09:17:31.544 INFO:teuthology.orchestra.run.smithi032.stdout:146 writesame 0x952bc5d thru 0x9539a87 data_size 0x5(0xde2b bytes) 2024-09-13T09:17:31.544 INFO:teuthology.orchestra.run.smithi032.stdout:148 write 0x224762a thru 0x22483c4 (0xd9b bytes) 2024-09-13T09:17:31.547 INFO:teuthology.orchestra.run.smithi032.stdout:150 write 0x28f91b3 thru 0x29012ff (0x814d bytes) 2024-09-13T09:17:31.562 INFO:teuthology.orchestra.run.smithi032.stdout:151 trunc from 0x9539a88 to 0x2fb092d 2024-09-13T09:17:31.563 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:31.560+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:31.596 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:17:31.596 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:17:31.596 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T09:17:31.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:17:31.621 INFO:teuthology.orchestra.run.smithi032.stdout:152 write 0xbb682ae thru 0xbb71daf (0x9b02 bytes) 2024-09-13T09:17:31.629 INFO:teuthology.orchestra.run.smithi032.stdout:155 punch from 0x9dc3574 to 0x9dc9320, (0x5dac bytes) 2024-09-13T09:17:31.629 INFO:teuthology.orchestra.run.smithi032.stdout:157 read 0x2d88149 thru 0x2d97979 (0xf831 bytes) 2024-09-13T09:17:31.630 INFO:teuthology.orchestra.run.smithi032.stdout:159 writesame 0x94598b3 thru 0x945ccf3 data_size 0x31(0x3441 bytes) 2024-09-13T09:17:31.636 INFO:teuthology.orchestra.run.smithi032.stdout:160 read 0x220617 thru 0x225f83 (0x596d bytes) 2024-09-13T09:17:31.637 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:31.637 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:31.653 INFO:teuthology.orchestra.run.smithi032.stdout:161 write 0xeb941bd thru 0xeb96017 (0x1e5b bytes) 2024-09-13T09:17:31.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:17:31.660 INFO:teuthology.orchestra.run.smithi032.stdout:162 punch from 0xb79cf3 to 0xb7a3f5, (0x702 bytes) 2024-09-13T09:17:31.660 INFO:teuthology.orchestra.run.smithi032.stdout:163 write 0xaf30fa6 thru 0xaf3f33e (0xe399 bytes) 2024-09-13T09:17:31.664 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:31.658+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:31.689 INFO:teuthology.orchestra.run.smithi032.stdout:166 trunc from 0xeb96018 to 0xc999b0 2024-09-13T09:17:31.716 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:17:31.728 INFO:teuthology.orchestra.run.smithi032.stdout:168 read 0x8663b3 thru 0x86d94c (0x759a bytes) 2024-09-13T09:17:31.745 INFO:teuthology.orchestra.run.smithi032.stdout:171 writesame 0x3f40806 thru 0x3f411b3 data_size 0x15(0x9ae bytes) 2024-09-13T09:17:31.745 INFO:teuthology.orchestra.run.smithi032.stdout:172 write 0x3059e45 thru 0x3069e40 (0xfffc bytes) 2024-09-13T09:17:31.751 INFO:teuthology.orchestra.run.smithi032.stdout:173 read 0x175dc66 thru 0x1763017 (0x53b2 bytes) 2024-09-13T09:17:31.752 INFO:teuthology.orchestra.run.smithi032.stdout:176 write 0x175b179 thru 0x1761cd2 (0x6b5a bytes) 2024-09-13T09:17:31.761 INFO:teuthology.orchestra.run.smithi032.stdout:182 trunc from 0x3f411b4 to 0x6b32c4e 2024-09-13T09:17:31.765 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:31.758+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:31.789 INFO:teuthology.orchestra.run.smithi032.stdout:184 write 0x9170e9b thru 0x917e99a (0xdb00 bytes) 2024-09-13T09:17:31.794 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:31.794 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:31.795 INFO:teuthology.orchestra.run.smithi032.stdout:185 writesame 0x4cbd17f thru 0x4ccb86b data_size 0x1(0xe6ed bytes) 2024-09-13T09:17:31.811 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:17:31.841 INFO:teuthology.orchestra.run.smithi032.stdout:186 write 0x9e9df0d thru 0x9ea46d5 (0x67c9 bytes) 2024-09-13T09:17:31.845 INFO:teuthology.orchestra.run.smithi032.stdout:188 punch from 0x4797305 to 0x4798c1d, (0x1918 bytes) 2024-09-13T09:17:31.907 INFO:teuthology.orchestra.run.smithi032.stdout:189 punch from 0x70c1a2a to 0x70c3a9d, (0x2073 bytes) 2024-09-13T09:17:31.907 INFO:teuthology.orchestra.run.smithi032.stdout:190 punch from 0x2325946 to 0x232c896, (0x6f50 bytes) 2024-09-13T09:17:31.908 INFO:teuthology.orchestra.run.smithi032.stdout:191 writesame 0xd9c4fcc thru 0xd9c777e data_size 0x1(0x27b3 bytes) 2024-09-13T09:17:31.908 INFO:teuthology.orchestra.run.smithi032.stdout:193 writesame 0xdc6dfeb thru 0xdc77995 data_size 0x1f(0x99ab bytes) 2024-09-13T09:17:31.908 INFO:teuthology.orchestra.run.smithi032.stdout:194 writesame 0x11eaf44 thru 0x11faf20 data_size 0x1(0xffdd bytes) 2024-09-13T09:17:31.909 INFO:teuthology.orchestra.run.smithi032.stdout:196 write 0xe30390 thru 0xe33d67 (0x39d8 bytes) 2024-09-13T09:17:31.909 INFO:teuthology.orchestra.run.smithi032.stdout:198 punch from 0xa452363 to 0xa4609a2, (0xe63f bytes) 2024-09-13T09:17:31.909 INFO:teuthology.orchestra.run.smithi032.stdout:199 punch from 0x2234223 to 0x2235898, (0x1675 bytes) 2024-09-13T09:17:31.909 INFO:teuthology.orchestra.run.smithi032.stdout:200 writesame 0x5ae31fc thru 0x5aedad4 data_size 0x5f(0xa8d9 bytes) 2024-09-13T09:17:31.909 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:31.904+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:31.910 INFO:teuthology.orchestra.run.smithi032.stdout:201 write 0x1b80dbf thru 0x1b826a4 (0x18e6 bytes) 2024-09-13T09:17:31.917 INFO:teuthology.orchestra.run.smithi032.stdout:202 read 0x723e827 thru 0x7243021 (0x47fb bytes) 2024-09-13T09:17:31.917 INFO:teuthology.orchestra.run.smithi032.stdout:203 punch from 0x49323f5 to 0x493e5ae, (0xc1b9 bytes) 2024-09-13T09:17:31.917 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:31.918 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xc428110 2024-09-13T09:17:31.919 INFO:teuthology.orchestra.run.smithi032.stdout:205 trunc from 0xdc77996 to 0xc428110 2024-09-13T09:17:31.931 INFO:teuthology.orchestra.run.smithi032.stdout:207 read 0x7552e11 thru 0x755f1aa (0xc39a bytes) 2024-09-13T09:17:31.931 INFO:teuthology.orchestra.run.smithi032.stdout:209 punch from 0x598c745 to 0x5997406, (0xacc1 bytes) 2024-09-13T09:17:31.931 INFO:teuthology.orchestra.run.smithi032.stdout:210 read 0x78fb91d thru 0x78fdcbf (0x23a3 bytes) 2024-09-13T09:17:31.932 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xea32e97 2024-09-13T09:17:31.935 INFO:teuthology.orchestra.run.smithi032.stdout:212 trunc from 0xc428110 to 0xea32e97 2024-09-13T09:17:31.946 INFO:teuthology.orchestra.run.smithi032.stdout:216 read 0x13ad7ae thru 0x13b7193 (0x99e6 bytes) 2024-09-13T09:17:31.947 INFO:teuthology.orchestra.run.smithi032.stdout:217 writesame 0x86f8c12 thru 0x870405b data_size 0x2f(0xb44a bytes) 2024-09-13T09:17:31.954 INFO:teuthology.orchestra.run.smithi032.stdout:218 read 0x52f7cac thru 0x52f8a18 (0xd6d bytes) 2024-09-13T09:17:31.954 INFO:teuthology.orchestra.run.smithi032.stdout:219 punch from 0x419e672 to 0x41ac78d, (0xe11b bytes) 2024-09-13T09:17:31.954 INFO:teuthology.orchestra.run.smithi032.stdout:220 writesame 0x4f93000 thru 0x4f99165 data_size 0x5b(0x6166 bytes) 2024-09-13T09:17:31.955 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:31.955 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:31.956 INFO:teuthology.orchestra.run.smithi032.stdout:221 read 0x35824bb thru 0x3582689 (0x1cf bytes) 2024-09-13T09:17:31.956 INFO:teuthology.orchestra.run.smithi032.stdout:223 read 0x54501a2 thru 0x545b007 (0xae66 bytes) 2024-09-13T09:17:31.956 INFO:teuthology.orchestra.run.smithi032.stdout:224 write 0xd29ba80 thru 0xd2a3591 (0x7b12 bytes) 2024-09-13T09:17:31.962 INFO:teuthology.orchestra.run.smithi032.stdout:225 write 0xc311c31 thru 0xc31899a (0x6d6a bytes) 2024-09-13T09:17:31.969 INFO:teuthology.orchestra.run.smithi032.stdout:226 read 0x42b265c thru 0x42b5241 (0x2be6 bytes) 2024-09-13T09:17:31.969 INFO:teuthology.orchestra.run.smithi032.stdout:229 punch from 0x89cfa5b to 0x89d4721, (0x4cc6 bytes) 2024-09-13T09:17:31.969 INFO:teuthology.orchestra.run.smithi032.stdout:231 read 0x8105d2 thru 0x818a2e (0x845d bytes) 2024-09-13T09:17:31.970 INFO:teuthology.orchestra.run.smithi032.stdout:232 read 0x1fee1d thru 0x200f34 (0x2118 bytes) 2024-09-13T09:17:31.970 INFO:teuthology.orchestra.run.smithi032.stdout:233 read 0xa4883a8 thru 0xa48c962 (0x45bb bytes) 2024-09-13T09:17:31.970 INFO:teuthology.orchestra.run.smithi032.stdout:235 read 0x1273efb thru 0x12748e4 (0x9ea bytes) 2024-09-13T09:17:31.971 INFO:teuthology.orchestra.run.smithi032.stdout:236 write 0x1d83f75 thru 0x1d8f8f1 (0xb97d bytes) 2024-09-13T09:17:31.972 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:17:31.976 INFO:teuthology.orchestra.run.smithi032.stdout:239 writesame 0xd658f1b thru 0xd65d7fa data_size 0x58(0x48e0 bytes) 2024-09-13T09:17:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:17:31.983 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 27, 2024-09-13T09:17:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "f0889cb1-03d6-4b81-88c9-6141846c6033", 2024-09-13T09:17:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-13T09:16:56.964982+0000", 2024-09-13T09:17:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-13T09:17:31.673658+0000", 2024-09-13T09:17:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-13T09:17:08.368378+0000", 2024-09-13T09:17:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-13T09:17:26.547451+0000", 2024-09-13T09:17:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:17:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5806080, 2024-09-13T09:17:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-13T09:17:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "nodeep-scrub", 2024-09-13T09:17:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "noscrub", 2024-09-13T09:17:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-13T09:17:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-13T09:17:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-13T09:17:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-13T09:17:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:17:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-09-13T09:17:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-13T09:17:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-13T09:17:10.140920+0000", 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-13T09:17:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-13T09:17:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "17", 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-13T09:17:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-13T09:17:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-13T09:17:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-13T09:17:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-13T09:17:15.775544+0000", 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-13T09:17:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-13T09:17:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "21", 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:17:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 21, 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-13T09:17:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-13T09:17:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-13T09:17:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:17:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:17:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:17:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-13T09:17:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-13T09:17:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-13T09:17:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-13T09:17:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-13T09:17:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-13T09:17:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-13T09:17:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-13T09:17:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:17:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:17:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:17:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:17:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-13T09:17:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-13T09:17:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-13T09:17:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-13T09:17:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-13T09:17:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-13T09:17:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.9900000095367432, 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.9900000095367432, 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-13T09:17:25.804931+0000", 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:17:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-13T09:17:32.000 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-13T09:17:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "25", 2024-09-13T09:17:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-13T09:17:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:17:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:17:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-13T09:17:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:17:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-13T09:17:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 25, 2024-09-13T09:17:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-13T09:17:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-13T09:17:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-13T09:17:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-13T09:17:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.7400000095367432, 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.7400000095367432, 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-13T09:17:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:17:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-09-13T09:17:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:17:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:17:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-13T09:17:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-13T09:17:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "786bdfc4-cc25-4167-aa5b-3737cfeb4ac1", 2024-09-13T09:17:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:17:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 0, 2024-09-13T09:17:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0, 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 0, 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-13T09:17:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:17:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-13T09:17:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:17:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:17:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-13T09:17:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:17:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-13T09:17:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:17:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/3549111487", 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/3549111487", 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/3549111487", 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/3549111487", 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "ca6189ab-90dd-4e9d-a85f-092186153b5a", 2024-09-13T09:17:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 22, 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-13T09:17:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:17:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-13T09:17:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:17:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:17:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:17:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6810/1470371899", 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6815/1470371899", 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1470371899", 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6820/1470371899", 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a", 2024-09-13T09:17:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 18, 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:17:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/2485123706", 2024-09-13T09:17:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/2485123706", 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/2485123706", 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/2485123706", 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218", 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:17:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 22, 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-13T09:17:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/1009890408", 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6814/1009890408", 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6821/1009890408", 2024-09-13T09:17:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6818/1009890408", 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39f42e87-29dd-42cc-9ef6-e0074f1794e4", 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:17:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 22, 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:17:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-13T09:17:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-13T09:17:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6823/2723023689", 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6825/2723023689", 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/2723023689", 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6828/2723023689", 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:17:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "cf8dd701-9543-4bdd-a6f6-0b56210f00c2", 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:17:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 22, 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6809/2955019640", 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6812/2955019640", 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6819/2955019640", 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6815/2955019640", 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:17:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "dd080b88-3292-4a34-8dbd-645683d8fe14", 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:17:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 22, 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-13T09:17:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-13T09:17:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:17:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:17:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-13T09:17:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:17:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6813/916160151", 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6818/916160151", 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6829/916160151", 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6822/916160151", 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:17:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c6bca54b-e982-4d6a-9ae9-057ee3881a5b", 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:17:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 25, 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-13T09:17:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:17:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:17:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-13T09:17:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:17:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:17:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-13T09:17:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:17:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:17:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-13T09:17:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:17:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6801/3477167689", 2024-09-13T09:17:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6803/3477167689", 2024-09-13T09:17:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6807/3477167689", 2024-09-13T09:17:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6805/3477167689", 2024-09-13T09:17:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:17:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:17:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:17:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:17:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-13T09:17:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-13T09:17:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:17:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:17:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:17:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:17:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 65536, 2024-09-13T09:17:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.247113+0000", 2024-09-13T09:17:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:17:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-13T09:17:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:17:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:17:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:17:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:17:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:17:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.166872+0000", 2024-09-13T09:17:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:17:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-13T09:17:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:17:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:17:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:17:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:17:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:17:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.451769+0000", 2024-09-13T09:17:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:17:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-13T09:17:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:17:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:17:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:17:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:17:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:17:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.458130+0000", 2024-09-13T09:17:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:17:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-13T09:17:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:17:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:17:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:17:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:17:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:17:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.466129+0000", 2024-09-13T09:17:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.444828+0000", 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.425145+0000", 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:17:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.484371+0000", 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-13T09:17:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-13T09:17:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-13T09:17:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-13T09:17:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-13T09:17:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-13T09:17:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-13T09:17:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-13T09:17:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-09-13T09:17:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-09-13T09:17:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [ 2024-09-13T09:17:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 2, 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-13T09:17:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-13T09:17:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-13T09:17:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:17:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:17:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-13T09:17:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:32.063 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:32.064 INFO:teuthology.orchestra.run.smithi032.stdout:240 trunc from 0xea32e97 to 0x2ddcc47 2024-09-13T09:17:32.065 INFO:tasks.thrashosds.thrasher:j is {'epoch': 27, 'fsid': 'f0889cb1-03d6-4b81-88c9-6141846c6033', 'created': '2024-09-13T09:16:56.964982+0000', 'modified': '2024-09-13T09:17:31.673658+0000', 'last_up_change': '2024-09-13T09:17:08.368378+0000', 'last_in_change': '2024-09-13T09:17:26.547451+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:17:10.140920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:17:15.775544+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:17:25.804931+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 25, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '786bdfc4-cc25-4167-aa5b-3737cfeb4ac1', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 3549111487}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 3549111487}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 3549111487}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 3549111487}]}, 'public_addr': '172.21.15.32:6801/3549111487', 'cluster_addr': '172.21.15.32:6803/3549111487', 'heartbeat_back_addr': '172.21.15.32:6807/3549111487', 'heartbeat_front_addr': '172.21.15.32:6805/3549111487', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'ca6189ab-90dd-4e9d-a85f-092186153b5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6810', 'nonce': 1470371899}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 1470371899}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1470371899}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6817', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6820', 'nonce': 1470371899}]}, 'public_addr': '172.21.15.32:6810/1470371899', 'cluster_addr': '172.21.15.32:6815/1470371899', 'heartbeat_back_addr': '172.21.15.32:6823/1470371899', 'heartbeat_front_addr': '172.21.15.32:6820/1470371899', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 2485123706}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 2485123706}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 2485123706}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 2485123706}]}, 'public_addr': '172.21.15.32:6825/2485123706', 'cluster_addr': '172.21.15.32:6827/2485123706', 'heartbeat_back_addr': '172.21.15.32:6831/2485123706', 'heartbeat_front_addr': '172.21.15.32:6829/2485123706', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6809', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 1009890408}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6813', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6814', 'nonce': 1009890408}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6819', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1009890408}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6818', 'nonce': 1009890408}]}, 'public_addr': '172.21.15.32:6811/1009890408', 'cluster_addr': '172.21.15.32:6814/1009890408', 'heartbeat_back_addr': '172.21.15.32:6821/1009890408', 'heartbeat_front_addr': '172.21.15.32:6818/1009890408', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39f42e87-29dd-42cc-9ef6-e0074f1794e4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6823', 'nonce': 2723023689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6824', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 2723023689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 2723023689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 2723023689}]}, 'public_addr': '172.21.15.46:6823/2723023689', 'cluster_addr': '172.21.15.46:6825/2723023689', 'heartbeat_back_addr': '172.21.15.46:6831/2723023689', 'heartbeat_front_addr': '172.21.15.46:6828/2723023689', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'cf8dd701-9543-4bdd-a6f6-0b56210f00c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6809', 'nonce': 2955019640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6811', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6812', 'nonce': 2955019640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 2955019640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6814', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 2955019640}]}, 'public_addr': '172.21.15.46:6809/2955019640', 'cluster_addr': '172.21.15.46:6812/2955019640', 'heartbeat_back_addr': '172.21.15.46:6819/2955019640', 'heartbeat_front_addr': '172.21.15.46:6815/2955019640', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'dd080b88-3292-4a34-8dbd-645683d8fe14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6810', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6813', 'nonce': 916160151}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 916160151}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 916160151}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 916160151}]}, 'public_addr': '172.21.15.46:6813/916160151', 'cluster_addr': '172.21.15.46:6818/916160151', 'heartbeat_back_addr': '172.21.15.46:6829/916160151', 'heartbeat_front_addr': '172.21.15.46:6822/916160151', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c6bca54b-e982-4d6a-9ae9-057ee3881a5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6801', 'nonce': 3477167689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6802', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 3477167689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6806', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 3477167689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6805', 'nonce': 3477167689}]}, 'public_addr': '172.21.15.46:6801/3477167689', 'cluster_addr': '172.21.15.46:6803/3477167689', 'heartbeat_back_addr': '172.21.15.46:6807/3477167689', 'heartbeat_front_addr': '172.21.15.46:6805/3477167689', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.247113+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.166872+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.451769+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.458130+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.466129+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.444828+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.425145+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.484371+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:17:32.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:17:32.066 INFO:teuthology.orchestra.run.smithi032.stdout:241 read 0x2c724db thru 0x2c7dc92 (0xb7b8 bytes) 2024-09-13T09:17:32.066 INFO:teuthology.orchestra.run.smithi032.stdout:242 write 0x26ca5d1 thru 0x26d3c63 (0x9693 bytes) 2024-09-13T09:17:32.067 INFO:teuthology.orchestra.run.smithi032.stdout:243 write 0xe35e648 thru 0xe35ead7 (0x490 bytes) 2024-09-13T09:17:32.067 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:32.067 INFO:teuthology.orchestra.run.smithi032.stdout:248 write 0xe46dab9 thru 0xe4766e5 (0x8c2d bytes) 2024-09-13T09:17:32.067 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:32.058+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:32.086 INFO:teuthology.orchestra.run.smithi032.stdout:249 trunc from 0xe4766e6 to 0x5151ca8 2024-09-13T09:17:32.105 INFO:teuthology.orchestra.run.smithi032.stdout:250 write 0x3f55872 thru 0x3f5f7b6 (0x9f45 bytes) 2024-09-13T09:17:32.106 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:32.106 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:32.121 INFO:teuthology.orchestra.run.smithi032.stdout:252 trunc from 0x5151ca8 to 0xc77c4f6 2024-09-13T09:17:32.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:17:32.129 INFO:teuthology.orchestra.run.smithi032.stdout:254 writesame 0x8dd706d thru 0x8de518a data_size 0x71(0xe11e bytes) 2024-09-13T09:17:32.134 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:32.134 INFO:teuthology.orchestra.run.smithi032.stdout:258 punch from 0x602fbcb to 0x603c0d6, (0xc50b bytes) 2024-09-13T09:17:32.149 INFO:teuthology.orchestra.run.smithi032.stdout:259 trunc from 0xc77c4f6 to 0x4620eb8 2024-09-13T09:17:32.166 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:32.162+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:32.168 INFO:teuthology.orchestra.run.smithi032.stdout:261 trunc from 0x4620eb8 to 0x91ec3af 2024-09-13T09:17:32.186 INFO:teuthology.orchestra.run.smithi032.stdout:262 trunc from 0x91ec3af to 0x3a7842a 2024-09-13T09:17:32.202 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:32.224 INFO:teuthology.orchestra.run.smithi032.stdout:264 writesame 0xdd47028 thru 0xdd55286 data_size 0x9(0xe25f bytes) 2024-09-13T09:17:32.224 INFO:teuthology.orchestra.run.smithi032.stdout:265 read 0x1f6ae66 thru 0x1f76f9e (0xc139 bytes) 2024-09-13T09:17:32.238 INFO:teuthology.orchestra.run.smithi032.stdout:267 trunc from 0xdd55287 to 0x481bd20 2024-09-13T09:17:32.265 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:32.262+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:32.267 INFO:teuthology.orchestra.run.smithi032.stdout:268 write 0xaf19031 thru 0xaf22c26 (0x9bf6 bytes) 2024-09-13T09:17:32.273 INFO:teuthology.orchestra.run.smithi032.stdout:270 read 0xa519fd0 thru 0xa51ad08 (0xd39 bytes) 2024-09-13T09:17:32.273 INFO:teuthology.orchestra.run.smithi032.stdout:275 read 0x7b4f46 thru 0x7bb979 (0x6a34 bytes) 2024-09-13T09:17:32.274 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:32.288 INFO:teuthology.orchestra.run.smithi032.stdout:278 write 0xdccd329 thru 0xdcdb560 (0xe238 bytes) 2024-09-13T09:17:32.294 INFO:teuthology.orchestra.run.smithi032.stdout:279 writesame 0x5d177e8 thru 0x5d1bc21 data_size 0x6(0x443a bytes) 2024-09-13T09:17:32.302 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:32.302 INFO:teuthology.orchestra.run.smithi032.stdout:282 write 0x8d3431a thru 0x8d372eb (0x2fd2 bytes) 2024-09-13T09:17:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:32.309 INFO:teuthology.orchestra.run.smithi032.stdout:283 write 0x1be5039 thru 0x1bf33aa (0xe372 bytes) 2024-09-13T09:17:32.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:17:32.323 INFO:teuthology.orchestra.run.smithi032.stdout:284 write 0xe057eae thru 0xe067c5c (0xfdaf bytes) 2024-09-13T09:17:32.329 INFO:teuthology.orchestra.run.smithi032.stdout:285 read 0x57e364b thru 0x57ea6c6 (0x707c bytes) 2024-09-13T09:17:32.329 INFO:teuthology.orchestra.run.smithi032.stdout:287 read 0xc2243ce thru 0xc2297b8 (0x53eb bytes) 2024-09-13T09:17:32.329 INFO:teuthology.orchestra.run.smithi032.stdout:288 punch from 0x3deb420 to 0x3df7ec5, (0xcaa5 bytes) 2024-09-13T09:17:32.329 INFO:teuthology.orchestra.run.smithi032.stdout:289 read 0xda21fe thru 0xdac957 (0xa75a bytes) 2024-09-13T09:17:32.330 INFO:teuthology.orchestra.run.smithi032.stdout:291 read 0x871b4ca thru 0x871b6f4 (0x22b bytes) 2024-09-13T09:17:32.330 INFO:teuthology.orchestra.run.smithi032.stdout:292 write 0x50a6183 thru 0x50ac216 (0x6094 bytes) 2024-09-13T09:17:32.336 INFO:teuthology.orchestra.run.smithi032.stdout:293 read 0x612aa49 thru 0x6138791 (0xdd49 bytes) 2024-09-13T09:17:32.336 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:32.336 INFO:teuthology.orchestra.run.smithi032.stdout:295 writesame 0x4723b61 thru 0x47272e4 data_size 0x22(0x3784 bytes) 2024-09-13T09:17:32.342 INFO:teuthology.orchestra.run.smithi032.stdout:296 write 0x4129877 thru 0x4130740 (0x6eca bytes) 2024-09-13T09:17:32.348 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:32.348 INFO:teuthology.orchestra.run.smithi032.stdout:298 write 0x3a6e8b1 thru 0x3a72cc5 (0x4415 bytes) 2024-09-13T09:17:32.366 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:32.364+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:32.379 INFO:teuthology.orchestra.run.smithi032.stdout:301 trunc from 0xe067c5d to 0x30c3761 2024-09-13T09:17:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:17:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:17:32.395 INFO:teuthology.orchestra.run.smithi032.stdout:302 read 0x14e3772 thru 0x14e6dfe (0x368d bytes) 2024-09-13T09:17:32.406 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-13T09:17:32.066768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590452,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":150,"num_read_kb":126,"num_write":167,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":219672,"kb_used_data":1824,"kb_used_omap":10,"kb_used_meta":217717,"kb_avail":660383208,"statfs":{"total":676457349120,"available":676232404992,"internally_reserved":0,"allocated":1867776,"data_stored":1814063,"data_compressed":19717,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":11150,"internal_metadata":222942322},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.389166"},"pg_stats":[{"pgid":"3.1f","version":"25'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.913416+0000","last_change":"2024-09-13T09:17:28.691093+0000","last_active":"2024-09-13T09:17:28.913416+0000","last_peered":"2024-09-13T09:17:28.913416+0000","last_clean":"2024-09-13T09:17:28.913416+0000","last_became_active":"2024-09-13T09:17:27.075690+0000","last_became_peered":"2024-09-13T09:17:27.075690+0000","last_unstale":"2024-09-13T09:17:28.913416+0000","last_undegraded":"2024-09-13T09:17:28.913416+0000","last_fullsized":"2024-09-13T09:17:28.913416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:24:54.534358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046778999999999999,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.684858+0000","last_change":"2024-09-13T09:17:28.684858+0000","last_active":"2024-09-13T09:17:28.684858+0000","last_peered":"2024-09-13T09:17:28.684858+0000","last_clean":"2024-09-13T09:17:28.684858+0000","last_became_active":"2024-09-13T09:17:26.697387+0000","last_became_peered":"2024-09-13T09:17:26.697387+0000","last_unstale":"2024-09-13T09:17:28.684858+0000","last_undegraded":"2024-09-13T09:17:28.684858+0000","last_fullsized":"2024-09-13T09:17:28.684858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:03.261921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382264,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.692127+0000","last_change":"2024-09-13T09:17:28.692127+0000","last_active":"2024-09-13T09:17:28.692127+0000","last_peered":"2024-09-13T09:17:28.692127+0000","last_clean":"2024-09-13T09:17:28.692127+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:17:28.692127+0000","last_undegraded":"2024-09-13T09:17:28.692127+0000","last_fullsized":"2024-09-13T09:17:28.692127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:20.509991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001448682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"25'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.920312+0000","last_change":"2024-09-13T09:17:28.691986+0000","last_active":"2024-09-13T09:17:28.920312+0000","last_peered":"2024-09-13T09:17:28.920312+0000","last_clean":"2024-09-13T09:17:28.920312+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:17:28.920312+0000","last_undegraded":"2024-09-13T09:17:28.920312+0000","last_fullsized":"2024-09-13T09:17:28.920312+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:05.002231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013280939999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.680836+0000","last_change":"2024-09-13T09:17:28.680836+0000","last_active":"2024-09-13T09:17:28.680836+0000","last_peered":"2024-09-13T09:17:28.680836+0000","last_clean":"2024-09-13T09:17:28.680836+0000","last_became_active":"2024-09-13T09:17:26.700541+0000","last_became_peered":"2024-09-13T09:17:26.700541+0000","last_unstale":"2024-09-13T09:17:28.680836+0000","last_undegraded":"2024-09-13T09:17:28.680836+0000","last_fullsized":"2024-09-13T09:17:28.680836+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:03.906254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042467999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"25'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.918213+0000","last_change":"2024-09-13T09:17:28.681288+0000","last_active":"2024-09-13T09:17:28.918213+0000","last_peered":"2024-09-13T09:17:28.918213+0000","last_clean":"2024-09-13T09:17:28.918213+0000","last_became_active":"2024-09-13T09:17:26.682478+0000","last_became_peered":"2024-09-13T09:17:26.682478+0000","last_unstale":"2024-09-13T09:17:28.918213+0000","last_undegraded":"2024-09-13T09:17:28.918213+0000","last_fullsized":"2024-09-13T09:17:28.918213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:29.916258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077588799999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.683318+0000","last_change":"2024-09-13T09:17:28.683318+0000","last_active":"2024-09-13T09:17:28.683318+0000","last_peered":"2024-09-13T09:17:28.683318+0000","last_clean":"2024-09-13T09:17:28.683318+0000","last_became_active":"2024-09-13T09:17:26.683581+0000","last_became_peered":"2024-09-13T09:17:26.683581+0000","last_unstale":"2024-09-13T09:17:28.683318+0000","last_undegraded":"2024-09-13T09:17:28.683318+0000","last_fullsized":"2024-09-13T09:17:28.683318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:13:18.827633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023026689999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.683153+0000","last_change":"2024-09-13T09:17:28.683153+0000","last_active":"2024-09-13T09:17:28.683153+0000","last_peered":"2024-09-13T09:17:28.683153+0000","last_clean":"2024-09-13T09:17:28.683153+0000","last_became_active":"2024-09-13T09:17:26.681831+0000","last_became_peered":"2024-09-13T09:17:26.681831+0000","last_unstale":"2024-09-13T09:17:28.683153+0000","last_undegraded":"2024-09-13T09:17:28.683153+0000","last_fullsized":"2024-09-13T09:17:28.683153+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:36:03.290864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001136335,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.692708+0000","last_change":"2024-09-13T09:17:28.692708+0000","last_active":"2024-09-13T09:17:28.692708+0000","last_peered":"2024-09-13T09:17:28.692708+0000","last_clean":"2024-09-13T09:17:28.692708+0000","last_became_active":"2024-09-13T09:17:26.697320+0000","last_became_peered":"2024-09-13T09:17:26.697320+0000","last_unstale":"2024-09-13T09:17:28.692708+0000","last_undegraded":"2024-09-13T09:17:28.692708+0000","last_fullsized":"2024-09-13T09:17:28.692708+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:12.823443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001841894,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.692331+0000","last_change":"2024-09-13T09:17:28.692331+0000","last_active":"2024-09-13T09:17:28.692331+0000","last_peered":"2024-09-13T09:17:28.692331+0000","last_clean":"2024-09-13T09:17:28.692331+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:17:28.692331+0000","last_undegraded":"2024-09-13T09:17:28.692331+0000","last_fullsized":"2024-09-13T09:17:28.692331+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:29.849557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016477340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.684144+0000","last_change":"2024-09-13T09:17:28.684144+0000","last_active":"2024-09-13T09:17:28.684144+0000","last_peered":"2024-09-13T09:17:28.684144+0000","last_clean":"2024-09-13T09:17:28.684144+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:17:28.684144+0000","last_undegraded":"2024-09-13T09:17:28.684144+0000","last_fullsized":"2024-09-13T09:17:28.684144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:10.717628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085639099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.681431+0000","last_change":"2024-09-13T09:17:28.681431+0000","last_active":"2024-09-13T09:17:28.681431+0000","last_peered":"2024-09-13T09:17:28.681431+0000","last_clean":"2024-09-13T09:17:28.681431+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:17:28.681431+0000","last_undegraded":"2024-09-13T09:17:28.681431+0000","last_fullsized":"2024-09-13T09:17:28.681431+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:42.341412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083138199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:17:27.772327+0000","last_change":"2024-09-13T09:17:19.574596+0000","last_active":"2024-09-13T09:17:27.772327+0000","last_peered":"2024-09-13T09:17:27.772327+0000","last_clean":"2024-09-13T09:17:27.772327+0000","last_became_active":"2024-09-13T09:17:17.495563+0000","last_became_peered":"2024-09-13T09:17:17.495563+0000","last_unstale":"2024-09-13T09:17:27.772327+0000","last_undegraded":"2024-09-13T09:17:27.772327+0000","last_fullsized":"2024-09-13T09:17:27.772327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:00.074904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096986200000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.684632+0000","last_change":"2024-09-13T09:17:28.684632+0000","last_active":"2024-09-13T09:17:28.684632+0000","last_peered":"2024-09-13T09:17:28.684632+0000","last_clean":"2024-09-13T09:17:28.684632+0000","last_became_active":"2024-09-13T09:17:26.685399+0000","last_became_peered":"2024-09-13T09:17:26.685399+0000","last_unstale":"2024-09-13T09:17:28.684632+0000","last_undegraded":"2024-09-13T09:17:28.684632+0000","last_fullsized":"2024-09-13T09:17:28.684632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:23:43.254812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002893765,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:17:27.686395+0000","last_change":"2024-09-13T09:17:19.504545+0000","last_active":"2024-09-13T09:17:27.686395+0000","last_peered":"2024-09-13T09:17:27.686395+0000","last_clean":"2024-09-13T09:17:27.686395+0000","last_became_active":"2024-09-13T09:17:17.494308+0000","last_became_peered":"2024-09-13T09:17:17.494308+0000","last_unstale":"2024-09-13T09:17:27.686395+0000","last_undegraded":"2024-09-13T09:17:27.686395+0000","last_fullsized":"2024-09-13T09:17:27.686395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:26.637575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503208,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.684573+0000","last_change":"2024-09-13T09:17:28.684573+0000","last_active":"2024-09-13T09:17:28.684573+0000","last_peered":"2024-09-13T09:17:28.684573+0000","last_clean":"2024-09-13T09:17:28.684573+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:17:28.684573+0000","last_undegraded":"2024-09-13T09:17:28.684573+0000","last_fullsized":"2024-09-13T09:17:28.684573+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:51.225828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086084099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:17:27.680790+0000","last_change":"2024-09-13T09:17:26.703969+0000","last_active":"2024-09-13T09:17:27.680790+0000","last_peered":"2024-09-13T09:17:27.680790+0000","last_clean":"2024-09-13T09:17:27.680790+0000","last_became_active":"2024-09-13T09:17:26.699011+0000","last_became_peered":"2024-09-13T09:17:26.699011+0000","last_unstale":"2024-09-13T09:17:27.680790+0000","last_undegraded":"2024-09-13T09:17:27.680790+0000","last_fullsized":"2024-09-13T09:17:27.680790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:34:04.861123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.683677+0000","last_change":"2024-09-13T09:17:28.683677+0000","last_active":"2024-09-13T09:17:28.683677+0000","last_peered":"2024-09-13T09:17:28.683677+0000","last_clean":"2024-09-13T09:17:28.683677+0000","last_became_active":"2024-09-13T09:17:27.078721+0000","last_became_peered":"2024-09-13T09:17:27.078721+0000","last_unstale":"2024-09-13T09:17:28.683677+0000","last_undegraded":"2024-09-13T09:17:28.683677+0000","last_fullsized":"2024-09-13T09:17:28.683677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:14.375967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024125790000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:17:27.686283+0000","last_change":"2024-09-13T09:17:26.701516+0000","last_active":"2024-09-13T09:17:27.686283+0000","last_peered":"2024-09-13T09:17:27.686283+0000","last_clean":"2024-09-13T09:17:27.686283+0000","last_became_active":"2024-09-13T09:17:26.701030+0000","last_became_peered":"2024-09-13T09:17:26.701030+0000","last_unstale":"2024-09-13T09:17:27.686283+0000","last_undegraded":"2024-09-13T09:17:27.686283+0000","last_fullsized":"2024-09-13T09:17:27.686283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:18.155724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.692726+0000","last_change":"2024-09-13T09:17:28.692726+0000","last_active":"2024-09-13T09:17:28.692726+0000","last_peered":"2024-09-13T09:17:28.692726+0000","last_clean":"2024-09-13T09:17:28.692726+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:17:28.692726+0000","last_undegraded":"2024-09-13T09:17:28.692726+0000","last_fullsized":"2024-09-13T09:17:28.692726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:00.161632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018814960000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.681160+0000","last_change":"2024-09-13T09:17:28.681160+0000","last_active":"2024-09-13T09:17:28.681160+0000","last_peered":"2024-09-13T09:17:28.681160+0000","last_clean":"2024-09-13T09:17:28.681160+0000","last_became_active":"2024-09-13T09:17:26.694930+0000","last_became_peered":"2024-09-13T09:17:26.694930+0000","last_unstale":"2024-09-13T09:17:28.681160+0000","last_undegraded":"2024-09-13T09:17:28.681160+0000","last_fullsized":"2024-09-13T09:17:28.681160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:20.826377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066593499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T09:17:26.686841+0000","last_change":"2024-09-13T09:17:19.509855+0000","last_active":"2024-09-13T09:17:26.686841+0000","last_peered":"2024-09-13T09:17:26.686841+0000","last_clean":"2024-09-13T09:17:26.686841+0000","last_became_active":"2024-09-13T09:17:17.495349+0000","last_became_peered":"2024-09-13T09:17:17.495349+0000","last_unstale":"2024-09-13T09:17:26.686841+0000","last_undegraded":"2024-09-13T09:17:26.686841+0000","last_fullsized":"2024-09-13T09:17:26.686841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:26:42.881389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.681404+0000","last_change":"2024-09-13T09:17:28.681404+0000","last_active":"2024-09-13T09:17:28.681404+0000","last_peered":"2024-09-13T09:17:28.681404+0000","last_clean":"2024-09-13T09:17:28.681404+0000","last_became_active":"2024-09-13T09:17:27.079067+0000","last_became_peered":"2024-09-13T09:17:27.079067+0000","last_unstale":"2024-09-13T09:17:28.681404+0000","last_undegraded":"2024-09-13T09:17:28.681404+0000","last_fullsized":"2024-09-13T09:17:28.681404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:10.671709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043683300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T09:17:27.076970+0000","last_change":"2024-09-13T09:17:19.575553+0000","last_active":"2024-09-13T09:17:27.076970+0000","last_peered":"2024-09-13T09:17:27.076970+0000","last_clean":"2024-09-13T09:17:27.076970+0000","last_became_active":"2024-09-13T09:17:17.493930+0000","last_became_peered":"2024-09-13T09:17:17.493930+0000","last_unstale":"2024-09-13T09:17:27.076970+0000","last_undegraded":"2024-09-13T09:17:27.076970+0000","last_fullsized":"2024-09-13T09:17:27.076970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:12.092799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000590198,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.680936+0000","last_change":"2024-09-13T09:17:28.680936+0000","last_active":"2024-09-13T09:17:28.680936+0000","last_peered":"2024-09-13T09:17:28.680936+0000","last_clean":"2024-09-13T09:17:28.680936+0000","last_became_active":"2024-09-13T09:17:26.683961+0000","last_became_peered":"2024-09-13T09:17:26.683961+0000","last_unstale":"2024-09-13T09:17:28.680936+0000","last_undegraded":"2024-09-13T09:17:28.680936+0000","last_fullsized":"2024-09-13T09:17:28.680936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:50:15.603364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052255699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:17:27.680346+0000","last_change":"2024-09-13T09:17:19.575190+0000","last_active":"2024-09-13T09:17:27.680346+0000","last_peered":"2024-09-13T09:17:27.680346+0000","last_clean":"2024-09-13T09:17:27.680346+0000","last_became_active":"2024-09-13T09:17:17.494196+0000","last_became_peered":"2024-09-13T09:17:17.494196+0000","last_unstale":"2024-09-13T09:17:27.680346+0000","last_undegraded":"2024-09-13T09:17:27.680346+0000","last_fullsized":"2024-09-13T09:17:27.680346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:50.071449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.686727+0000","last_change":"2024-09-13T09:17:28.686727+0000","last_active":"2024-09-13T09:17:28.686727+0000","last_peered":"2024-09-13T09:17:28.686727+0000","last_clean":"2024-09-13T09:17:28.686727+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:17:28.686727+0000","last_undegraded":"2024-09-13T09:17:28.686727+0000","last_fullsized":"2024-09-13T09:17:28.686727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:44.094850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061431400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T09:17:26.685978+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:17:26.685978+0000","last_peered":"2024-09-13T09:17:26.685978+0000","last_clean":"2024-09-13T09:17:26.685978+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:17:26.685978+0000","last_undegraded":"2024-09-13T09:17:26.685978+0000","last_fullsized":"2024-09-13T09:17:26.685978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":182,"reported_epoch":25,"state":"peering","last_fresh":"2024-09-13T09:17:28.684155+0000","last_change":"2024-09-13T09:17:28.684379+0000","last_active":"2024-09-13T09:17:28.684155+0000","last_peered":"2024-09-13T09:17:27.784211+0000","last_clean":"2024-09-13T09:17:19.574677+0000","last_became_active":"2024-09-13T09:17:27.775689+0000","last_became_peered":"2024-09-13T09:17:27.775689+0000","last_unstale":"2024-09-13T09:17:28.684155+0000","last_undegraded":"2024-09-13T09:17:28.684155+0000","last_fullsized":"2024-09-13T09:17:28.684155+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:15.926725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.683581+0000","last_change":"2024-09-13T09:17:28.683581+0000","last_active":"2024-09-13T09:17:28.683581+0000","last_peered":"2024-09-13T09:17:28.683581+0000","last_clean":"2024-09-13T09:17:28.683581+0000","last_became_active":"2024-09-13T09:17:26.681861+0000","last_became_peered":"2024-09-13T09:17:26.681861+0000","last_unstale":"2024-09-13T09:17:28.683581+0000","last_undegraded":"2024-09-13T09:17:28.683581+0000","last_fullsized":"2024-09-13T09:17:28.683581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:40:45.638490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014243249999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.681630+0000","last_change":"2024-09-13T09:17:28.681630+0000","last_active":"2024-09-13T09:17:28.681630+0000","last_peered":"2024-09-13T09:17:28.681630+0000","last_clean":"2024-09-13T09:17:28.681630+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:17:28.681630+0000","last_undegraded":"2024-09-13T09:17:28.681630+0000","last_fullsized":"2024-09-13T09:17:28.681630+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:06.071033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013508260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.687080+0000","last_change":"2024-09-13T09:17:28.687080+0000","last_active":"2024-09-13T09:17:28.687080+0000","last_peered":"2024-09-13T09:17:28.687080+0000","last_clean":"2024-09-13T09:17:28.687080+0000","last_became_active":"2024-09-13T09:17:26.711477+0000","last_became_peered":"2024-09-13T09:17:26.711477+0000","last_unstale":"2024-09-13T09:17:28.687080+0000","last_undegraded":"2024-09-13T09:17:28.687080+0000","last_fullsized":"2024-09-13T09:17:28.687080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:30.419716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033158900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.685897+0000","last_change":"2024-09-13T09:17:28.685897+0000","last_active":"2024-09-13T09:17:28.685897+0000","last_peered":"2024-09-13T09:17:28.685897+0000","last_clean":"2024-09-13T09:17:28.685897+0000","last_became_active":"2024-09-13T09:17:26.696189+0000","last_became_peered":"2024-09-13T09:17:26.696189+0000","last_unstale":"2024-09-13T09:17:28.685897+0000","last_undegraded":"2024-09-13T09:17:28.685897+0000","last_fullsized":"2024-09-13T09:17:28.685897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:17.482993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002522725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.906297+0000","last_change":"2024-09-13T09:17:28.693070+0000","last_active":"2024-09-13T09:17:28.906297+0000","last_peered":"2024-09-13T09:17:28.906297+0000","last_clean":"2024-09-13T09:17:28.906297+0000","last_became_active":"2024-09-13T09:17:26.698657+0000","last_became_peered":"2024-09-13T09:17:26.698657+0000","last_unstale":"2024-09-13T09:17:28.906297+0000","last_undegraded":"2024-09-13T09:17:28.906297+0000","last_fullsized":"2024-09-13T09:17:28.906297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:12.388714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002213343,"stat_sum":{"num_bytes":30,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.684477+0000","last_change":"2024-09-13T09:17:28.684477+0000","last_active":"2024-09-13T09:17:28.684477+0000","last_peered":"2024-09-13T09:17:28.684477+0000","last_clean":"2024-09-13T09:17:28.684477+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:17:28.684477+0000","last_undegraded":"2024-09-13T09:17:28.684477+0000","last_fullsized":"2024-09-13T09:17:28.684477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:25.411265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099816899999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.687062+0000","last_change":"2024-09-13T09:17:28.687062+0000","last_active":"2024-09-13T09:17:28.687062+0000","last_peered":"2024-09-13T09:17:28.687062+0000","last_clean":"2024-09-13T09:17:28.687062+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:17:28.687062+0000","last_undegraded":"2024-09-13T09:17:28.687062+0000","last_fullsized":"2024-09-13T09:17:28.687062+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:31.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034733699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.684481+0000","last_change":"2024-09-13T09:17:28.684481+0000","last_active":"2024-09-13T09:17:28.684481+0000","last_peered":"2024-09-13T09:17:28.684481+0000","last_clean":"2024-09-13T09:17:28.684481+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:17:28.684481+0000","last_undegraded":"2024-09-13T09:17:28.684481+0000","last_fullsized":"2024-09-13T09:17:28.684481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:56.423075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151104,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.687780+0000","last_change":"2024-09-13T09:17:28.687780+0000","last_active":"2024-09-13T09:17:28.687780+0000","last_peered":"2024-09-13T09:17:28.687780+0000","last_clean":"2024-09-13T09:17:28.687780+0000","last_became_active":"2024-09-13T09:17:26.709035+0000","last_became_peered":"2024-09-13T09:17:26.709035+0000","last_unstale":"2024-09-13T09:17:28.687780+0000","last_undegraded":"2024-09-13T09:17:28.687780+0000","last_fullsized":"2024-09-13T09:17:28.687780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:44:37.440254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037557999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.683788+0000","last_change":"2024-09-13T09:17:28.683788+0000","last_active":"2024-09-13T09:17:28.683788+0000","last_peered":"2024-09-13T09:17:28.683788+0000","last_clean":"2024-09-13T09:17:28.683788+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:17:28.683788+0000","last_undegraded":"2024-09-13T09:17:28.683788+0000","last_fullsized":"2024-09-13T09:17:28.683788+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:47.572839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070075400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.684054+0000","last_change":"2024-09-13T09:17:28.684054+0000","last_active":"2024-09-13T09:17:28.684054+0000","last_peered":"2024-09-13T09:17:28.684054+0000","last_clean":"2024-09-13T09:17:28.684054+0000","last_became_active":"2024-09-13T09:17:26.698560+0000","last_became_peered":"2024-09-13T09:17:26.698560+0000","last_unstale":"2024-09-13T09:17:28.684054+0000","last_undegraded":"2024-09-13T09:17:28.684054+0000","last_fullsized":"2024-09-13T09:17:28.684054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:24:25.246107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084252600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:17:28.683729+0000","last_change":"2024-09-13T09:17:28.683729+0000","last_active":"2024-09-13T09:17:28.683729+0000","last_peered":"2024-09-13T09:17:28.683729+0000","last_clean":"2024-09-13T09:17:28.683729+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:17:28.683729+0000","last_undegraded":"2024-09-13T09:17:28.683729+0000","last_fullsized":"2024-09-13T09:17:28.683729+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:07.893848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000594467,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":29056,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":311296,"data_stored":682591,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":294912,"data_stored":682591,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":245760,"data_stored":86098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":258048,"data_stored":92231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":253952,"data_stored":92223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":241664,"data_stored":86079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":262144,"data_stored":92250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":0,"up_from":0,"seq":55834574856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":30,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:17:32.409 INFO:tasks.thrashosds.thrasher:Setting 3.16 to [6, 2, 3, 4] 2024-09-13T09:17:32.409 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.16', '6', '2', '3', '4'] 2024-09-13T09:17:32.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.16 6 2 3 4 2024-09-13T09:17:32.422 INFO:teuthology.orchestra.run.smithi032.stdout:304 writesame 0xe2693d4 thru 0xe275f52 data_size 0x73(0xcb7f bytes) 2024-09-13T09:17:32.422 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:32.422 INFO:teuthology.orchestra.run.smithi032.stdout:306 read 0x3eff6c2 thru 0x3f0e26f (0xebae bytes) 2024-09-13T09:17:32.422 INFO:teuthology.orchestra.run.smithi032.stdout:307 write 0x69964d7 thru 0x69974d7 (0x1001 bytes) 2024-09-13T09:17:32.425 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:32.433 INFO:teuthology.orchestra.run.smithi032.stdout:309 write 0xe745868 thru 0xe755829 (0xffc2 bytes) 2024-09-13T09:17:32.437 INFO:teuthology.orchestra.run.smithi032.stdout:310 write 0x96a1b7b thru 0x96a6071 (0x44f7 bytes) 2024-09-13T09:17:32.442 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:32.442 INFO:teuthology.orchestra.run.smithi032.stdout:315 writesame 0xb773923 thru 0xb777bf1 data_size 0x1(0x42cf bytes) 2024-09-13T09:17:32.454 INFO:teuthology.orchestra.run.smithi032.stdout:316 read 0xd993c1a thru 0xd9959c4 (0x1dab bytes) 2024-09-13T09:17:32.454 INFO:teuthology.orchestra.run.smithi032.stdout:317 punch from 0x8028238 to 0x802e1f1, (0x5fb9 bytes) 2024-09-13T09:17:32.454 INFO:teuthology.orchestra.run.smithi032.stdout:319 read 0xa383ff8 thru 0xa38c78c (0x8795 bytes) 2024-09-13T09:17:32.454 INFO:teuthology.orchestra.run.smithi032.stdout:320 write 0xa60d5f9 thru 0xa612284 (0x4c8c bytes) 2024-09-13T09:17:32.466 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:32.464+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:32.482 INFO:teuthology.orchestra.run.smithi032.stdout:321 trunc from 0xe75582a to 0x5cd58b 2024-09-13T09:17:32.496 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:32.496 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:32.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:17:32.518 INFO:teuthology.orchestra.run.smithi032.stdout:322 trunc from 0x5cd58b to 0xb32b00c 2024-09-13T09:17:32.528 INFO:teuthology.orchestra.run.smithi032.stdout:323 punch from 0xacbd7ca to 0xacc3b08, (0x633e bytes) 2024-09-13T09:17:32.540 INFO:teuthology.orchestra.run.smithi032.stdout:324 write 0xc29a0b4 thru 0xc2a0d29 (0x6c76 bytes) 2024-09-13T09:17:32.550 INFO:teuthology.orchestra.run.smithi032.stdout:325 trunc from 0xc2a0d2a to 0x9b3dd44 2024-09-13T09:17:32.567 INFO:teuthology.orchestra.run.smithi032.stdout:326 punch from 0x203936b to 0x2042d3b, (0x99d0 bytes) 2024-09-13T09:17:32.567 INFO:teuthology.orchestra.run.smithi032.stdout:327 write 0x91cce6f thru 0x91d5661 (0x87f3 bytes) 2024-09-13T09:17:32.568 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:32.562+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:32.571 INFO:teuthology.orchestra.run.smithi032.stdout:329 write 0x63282a1 thru 0x6332a08 (0xa768 bytes) 2024-09-13T09:17:32.574 INFO:teuthology.orchestra.run.smithi032.stdout:330 write 0x704aa2b thru 0x7058cd5 (0xe2ab bytes) 2024-09-13T09:17:32.579 INFO:teuthology.orchestra.run.smithi032.stdout:331 read 0x62ddb3 thru 0x630fa8 (0x31f6 bytes) 2024-09-13T09:17:32.581 INFO:teuthology.orchestra.run.smithi032.stdout:332 trunc from 0x9b3dd44 to 0x84243bb 2024-09-13T09:17:32.596 INFO:teuthology.orchestra.run.smithi032.stdout:333 write 0x712ffbc thru 0x7130570 (0x5b5 bytes) 2024-09-13T09:17:32.598 INFO:teuthology.orchestra.run.smithi032.stdout:334 clone 1 order 22 su 524288 sc 7 2024-09-13T09:17:32.636 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:32.636 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:32.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:17:32.694 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:32.690+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:17:32.769 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:32.766+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:32.789 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:17:32.835 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:32.835 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:32.850 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:17:32.871 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:32.868+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:32.909 INFO:teuthology.orchestra.run.smithi032.stdout:335 punch from 0x5c42835 to 0x5c522be, (0xfa89 bytes) 2024-09-13T09:17:32.914 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:32.914 INFO:teuthology.orchestra.run.smithi032.stdout:338 write 0x4343947 thru 0x4353510 (0xfbca bytes) 2024-09-13T09:17:32.927 INFO:teuthology.orchestra.run.smithi032.stdout:339 write 0x8e209d9 thru 0x8e22eae (0x24d6 bytes) 2024-09-13T09:17:32.931 INFO:teuthology.orchestra.run.smithi032.stdout:340 write 0x131375c thru 0x1320c94 (0xd539 bytes) 2024-09-13T09:17:32.935 INFO:teuthology.orchestra.run.smithi032.stdout:341 read 0x38537dd thru 0x3859bc3 (0x63e7 bytes) 2024-09-13T09:17:32.935 INFO:teuthology.orchestra.run.smithi032.stdout:343 read 0x2935877 thru 0x293f372 (0x9afc bytes) 2024-09-13T09:17:32.941 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:32.941 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:32.942 INFO:teuthology.orchestra.run.smithi032.stdout:345 write 0x9320929 thru 0x9326aac (0x6184 bytes) 2024-09-13T09:17:32.947 INFO:teuthology.orchestra.run.smithi032.stdout:346 write 0x92b69ca thru 0x92bc634 (0x5c6b bytes) 2024-09-13T09:17:32.950 INFO:teuthology.orchestra.run.smithi032.stdout:347 writesame 0x304dee5 thru 0x3058b35 data_size 0x1(0xac51 bytes) 2024-09-13T09:17:32.958 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:32.959 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:32.972 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:32.968+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:32.975 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:32.975 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:32.991 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:32.991 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:32.992 INFO:teuthology.orchestra.run.smithi032.stdout:349 write 0x34a5092 thru 0x34adc43 (0x8bb2 bytes) 2024-09-13T09:17:32.997 INFO:teuthology.orchestra.run.smithi032.stdout:353 punch from 0x245bc6c to 0x246774c, (0xbae0 bytes) 2024-09-13T09:17:33.009 INFO:teuthology.orchestra.run.smithi032.stdout:356 trunc from 0x9326aad to 0xe9ce745 2024-09-13T09:17:33.011 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:33.012 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:33.022 INFO:teuthology.orchestra.run.smithi032.stdout:357 read 0x47596a8 thru 0x4762afd (0x9456 bytes) 2024-09-13T09:17:33.022 INFO:teuthology.orchestra.run.smithi032.stdout:359 writesame 0x6beffad thru 0x6bfb905 data_size 0x1(0xb959 bytes) 2024-09-13T09:17:33.032 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:33.038 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:33.038 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:33.053 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:33.053 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:33.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:17:33.071 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:33.071 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:33.073 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.073 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:33.068+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:33.086 INFO:teuthology.orchestra.run.smithi032.stdout:362 trunc from 0xe9ce745 to 0x6b9bb1b 2024-09-13T09:17:33.111 INFO:teuthology.orchestra.run.smithi032.stdout:364 punch from 0x22a67dd to 0x22a7670, (0xe93 bytes) 2024-09-13T09:17:33.111 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.111 INFO:teuthology.orchestra.run.smithi032.stdout:366 punch from 0xff452 to 0x109a3a, (0xa5e8 bytes) 2024-09-13T09:17:33.111 INFO:teuthology.orchestra.run.smithi032.stdout:367 read 0x27deabf thru 0x27e5640 (0x6b82 bytes) 2024-09-13T09:17:33.111 INFO:teuthology.orchestra.run.smithi032.stdout:368 read 0x591826c thru 0x591dc53 (0x59e8 bytes) 2024-09-13T09:17:33.128 INFO:teuthology.orchestra.run.smithi032.stdout:369 write 0xb4793db thru 0xb47df33 (0x4b59 bytes) 2024-09-13T09:17:33.133 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.133 INFO:teuthology.orchestra.run.smithi032.stdout:373 punch from 0x65c5a99 to 0x65ca761, (0x4cc8 bytes) 2024-09-13T09:17:33.139 INFO:teuthology.orchestra.run.smithi032.stdout:375 trunc from 0xb47df34 to 0x7e0a58d 2024-09-13T09:17:33.154 INFO:teuthology.orchestra.run.smithi032.stdout:376 writesame 0x74b62cf thru 0x74beecb data_size 0x1(0x8bfd bytes) 2024-09-13T09:17:33.174 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:33.170+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:33.188 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:33.188 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:33.188 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:17:33.188 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:17:33.189 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:33.191 INFO:teuthology.orchestra.run.smithi032.stdout:377 trunc from 0x7e0a58d to 0x81087d3 2024-09-13T09:17:33.203 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:17:33.204 INFO:teuthology.orchestra.run.smithi032.stdout:378 write 0x51f5447 thru 0x51fbfb4 (0x6b6e bytes) 2024-09-13T09:17:33.210 INFO:teuthology.orchestra.run.smithi032.stdout:379 write 0x6286411 thru 0x6291fad (0xbb9d bytes) 2024-09-13T09:17:33.213 INFO:teuthology.orchestra.run.smithi032.stdout:380 write 0x5588ca0 thru 0x55935f9 (0xa95a bytes) 2024-09-13T09:17:33.246 INFO:teuthology.orchestra.run.smithi032.stdout:381 write 0xe0e7f95 thru 0xe0edf49 (0x5fb5 bytes) 2024-09-13T09:17:33.268 INFO:teuthology.orchestra.run.smithi032.stdout:382 trunc from 0xe0edf4a to 0x91eeca5 2024-09-13T09:17:33.290 INFO:teuthology.orchestra.run.smithi032.stdout:383 read 0x32d7d96 thru 0x32e6e45 (0xf0b0 bytes) 2024-09-13T09:17:33.290 INFO:teuthology.orchestra.run.smithi032.stdout:384 read 0x569e449 thru 0x569ebf1 (0x7a9 bytes) 2024-09-13T09:17:33.291 INFO:teuthology.orchestra.run.smithi032.stdout:385 read 0xad7cef thru 0xadd983 (0x5c95 bytes) 2024-09-13T09:17:33.291 INFO:teuthology.orchestra.run.smithi032.stdout:386 read 0x6e38c79 thru 0x6e41b20 (0x8ea8 bytes) 2024-09-13T09:17:33.292 INFO:teuthology.orchestra.run.smithi032.stdout:388 read 0x49c7158 thru 0x49d57fa (0xe6a3 bytes) 2024-09-13T09:17:33.293 INFO:teuthology.orchestra.run.smithi032.stdout:390 write 0x329f139 thru 0x32af02e (0xfef6 bytes) 2024-09-13T09:17:33.298 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:33.294+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:33.300 INFO:teuthology.orchestra.run.smithi032.stdout:393 write 0x4ebe955 thru 0x4ec1a8a (0x3136 bytes) 2024-09-13T09:17:33.307 INFO:teuthology.orchestra.run.smithi032.stdout:395 write 0x66d37aa thru 0x66da55e (0x6db5 bytes) 2024-09-13T09:17:33.310 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.311 INFO:teuthology.orchestra.run.smithi032.stdout:397 read 0x671ef4 thru 0x675d7f (0x3e8c bytes) 2024-09-13T09:17:33.311 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.311 INFO:teuthology.orchestra.run.smithi032.stdout:399 write 0x881905b thru 0x8821a82 (0x8a28 bytes) 2024-09-13T09:17:33.318 INFO:teuthology.orchestra.run.smithi032.stdout:400 read 0x120eb32 thru 0x12122bf (0x378e bytes) 2024-09-13T09:17:33.318 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.318 INFO:teuthology.orchestra.run.smithi032.stdout:402 punch from 0x16d94df to 0x16de908, (0x5429 bytes) 2024-09-13T09:17:33.334 INFO:teuthology.orchestra.run.smithi032.stdout:403 writesame 0xa197b9f thru 0xa19eba6 data_size 0x3c(0x7008 bytes) 2024-09-13T09:17:33.334 INFO:teuthology.orchestra.run.smithi032.stdout:404 writesame 0x9c89a77 thru 0x9c8d31f data_size 0x5(0x38a9 bytes) 2024-09-13T09:17:33.342 INFO:teuthology.orchestra.run.smithi032.stdout:405 read 0x87ea319 thru 0x87f8b4d (0xe835 bytes) 2024-09-13T09:17:33.342 INFO:teuthology.orchestra.run.smithi032.stdout:406 writesame 0x66f8b6a thru 0x67049d7 data_size 0x27(0xbe6e bytes) 2024-09-13T09:17:33.359 INFO:teuthology.orchestra.run.smithi032.stdout:407 write 0xa27194e thru 0xa280352 (0xea05 bytes) 2024-09-13T09:17:33.368 INFO:teuthology.orchestra.run.smithi032.stdout:408 writesame 0x3d078cb thru 0x3d0ae33 data_size 0xb(0x3569 bytes) 2024-09-13T09:17:33.375 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:33.372+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:33.389 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:33.389 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:17:33.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:17:33.389 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [] 2024-09-13T09:17:33.389 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:33.391 INFO:teuthology.orchestra.run.smithi032.stdout:409 writesame 0xc41f4ea thru 0xc42be05 data_size 0x3d(0xc91c bytes) 2024-09-13T09:17:33.391 INFO:teuthology.orchestra.run.smithi032.stdout:410 write 0x13e2119 thru 0x13e9fa2 (0x7e8a bytes) 2024-09-13T09:17:33.399 INFO:teuthology.orchestra.run.smithi032.stdout:411 read 0x4040fb1 thru 0x404240d (0x145d bytes) 2024-09-13T09:17:33.400 INFO:teuthology.orchestra.run.smithi032.stdout:412 punch from 0xff5c6c to 0xff91c6, (0x355a bytes) 2024-09-13T09:17:33.400 INFO:teuthology.orchestra.run.smithi032.stdout:413 punch from 0x13e04ab to 0x13e30e4, (0x2c39 bytes) 2024-09-13T09:17:33.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:17:33.416 INFO:teuthology.orchestra.run.smithi032.stdout:414 write 0xca3f94f thru 0xca483b3 (0x8a65 bytes) 2024-09-13T09:17:33.423 INFO:teuthology.orchestra.run.smithi032.stdout:415 write 0x6c1c290 thru 0x6c2b873 (0xf5e4 bytes) 2024-09-13T09:17:33.446 INFO:teuthology.orchestra.run.smithi032.stdout:416 write 0xed3c23a thru 0xed45e66 (0x9c2d bytes) 2024-09-13T09:17:33.453 INFO:teuthology.orchestra.run.smithi032.stdout:417 write 0x38321a5 thru 0x383da9c (0xb8f8 bytes) 2024-09-13T09:17:33.461 INFO:teuthology.orchestra.run.smithi032.stdout:420 write 0x51ab0ab thru 0x51afeb1 (0x4e07 bytes) 2024-09-13T09:17:33.467 INFO:teuthology.orchestra.run.smithi032.stdout:421 trunc from 0xed45e67 to 0xdbda7cb 2024-09-13T09:17:33.476 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:33.472+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:33.489 INFO:teuthology.orchestra.run.smithi032.stdout:422 write 0x4865ec3 thru 0x4868790 (0x28ce bytes) 2024-09-13T09:17:33.495 INFO:teuthology.orchestra.run.smithi032.stdout:424 read 0x1349c15 thru 0x1358a69 (0xee55 bytes) 2024-09-13T09:17:33.496 INFO:teuthology.orchestra.run.smithi032.stdout:425 punch from 0xd673959 to 0xd682198, (0xe83f bytes) 2024-09-13T09:17:33.496 INFO:teuthology.orchestra.run.smithi032.stdout:426 write 0xc841ac0 thru 0xc8480ca (0x660b bytes) 2024-09-13T09:17:33.536 INFO:teuthology.orchestra.run.smithi032.stdout:427 trunc from 0xdbda7cb to 0x3de5e1 2024-09-13T09:17:33.577 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:33.572+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:33.577 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.579 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:17:33.580 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:33.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:17:33.602 INFO:teuthology.orchestra.run.smithi032.stdout:429 writesame 0x5978334 thru 0x597bba9 data_size 0x49(0x3876 bytes) 2024-09-13T09:17:33.623 INFO:teuthology.orchestra.run.smithi032.stdout:430 writesame 0x8ea3297 thru 0x8eabd0b data_size 0x5(0x8a75 bytes) 2024-09-13T09:17:33.623 INFO:teuthology.orchestra.run.smithi032.stdout:432 writesame 0x236805f thru 0x236f8da data_size 0xb(0x787c bytes) 2024-09-13T09:17:33.631 INFO:teuthology.orchestra.run.smithi032.stdout:433 punch from 0x8b9d4ea to 0x8ba0f4c, (0x3a62 bytes) 2024-09-13T09:17:33.653 INFO:teuthology.orchestra.run.smithi032.stdout:435 write 0xcd9af4e thru 0xcdaad82 (0xfe35 bytes) 2024-09-13T09:17:33.658 INFO:teuthology.orchestra.run.smithi032.stdout:437 writesame 0x7175de6 thru 0x717f4bb data_size 0x2b(0x96d6 bytes) 2024-09-13T09:17:33.664 INFO:teuthology.orchestra.run.smithi032.stdout:438 read 0xaab749 thru 0xab6543 (0xadfb bytes) 2024-09-13T09:17:33.665 INFO:teuthology.orchestra.run.smithi032.stdout:441 read 0xc171254 thru 0xc17b198 (0x9f45 bytes) 2024-09-13T09:17:33.665 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.665 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.665 INFO:teuthology.orchestra.run.smithi032.stdout:444 punch from 0x775135e to 0x775bb3b, (0xa7dd bytes) 2024-09-13T09:17:33.666 INFO:teuthology.orchestra.run.smithi032.stdout:446 writesame 0x2534f89 thru 0x253bf32 data_size 0x1(0x6faa bytes) 2024-09-13T09:17:33.677 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:33.670+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:33.689 INFO:teuthology.orchestra.run.smithi032.stdout:449 writesame 0x1a48efa thru 0x1a55c33 data_size 0x6d(0xcd3a bytes) 2024-09-13T09:17:33.693 INFO:teuthology.orchestra.run.smithi032.stdout:450 punch from 0x9e5b7c2 to 0x9e60bd8, (0x5416 bytes) 2024-09-13T09:17:33.693 INFO:teuthology.orchestra.run.smithi032.stdout:451 writesame 0x970b969 thru 0x970dd83 data_size 0x27(0x241b bytes) 2024-09-13T09:17:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:set 3.16 pg_upmap_items mapping to [6->2,3->4] 2024-09-13T09:17:33.725 INFO:teuthology.orchestra.run.smithi032.stdout:452 trunc from 0xcdaad83 to 0x47c8d3b 2024-09-13T09:17:33.739 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:33.739 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:33.739 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:17:33.739 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:17:33.739 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:33.751 INFO:teuthology.orchestra.run.smithi032.stdout:453 read 0x3713f67 thru 0x371feb9 (0xbf53 bytes) 2024-09-13T09:17:33.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:17:33.773 INFO:teuthology.orchestra.run.smithi032.stdout:454 write 0x91979db thru 0x919feb8 (0x84de bytes) 2024-09-13T09:17:33.778 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:33.776+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:33.788 INFO:teuthology.orchestra.run.smithi032.stdout:455 trunc from 0x919feb9 to 0xc700c5f 2024-09-13T09:17:33.801 INFO:teuthology.orchestra.run.smithi032.stdout:456 write 0xbd4a756 thru 0xbd514b5 (0x6d60 bytes) 2024-09-13T09:17:33.807 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.807 INFO:teuthology.orchestra.run.smithi032.stdout:458 punch from 0x7ebb519 to 0x7ec983b, (0xe322 bytes) 2024-09-13T09:17:33.808 INFO:teuthology.orchestra.run.smithi032.stdout:459 read 0x3fdd55 thru 0x40492c (0x6bd8 bytes) 2024-09-13T09:17:33.808 INFO:teuthology.orchestra.run.smithi032.stdout:460 punch from 0xc030b6f to 0xc03cdaf, (0xc240 bytes) 2024-09-13T09:17:33.808 INFO:teuthology.orchestra.run.smithi032.stdout:461 write 0x7765c32 thru 0x776f72e (0x9afd bytes) 2024-09-13T09:17:33.814 INFO:teuthology.orchestra.run.smithi032.stdout:463 write 0x34ebf69 thru 0x34f5201 (0x9299 bytes) 2024-09-13T09:17:33.816 INFO:teuthology.orchestra.run.smithi032.stdout:464 read 0x32294b thru 0x32acd0 (0x8386 bytes) 2024-09-13T09:17:33.818 INFO:teuthology.orchestra.run.smithi032.stdout:465 punch from 0x67a92e2 to 0x67b31b1, (0x9ecf bytes) 2024-09-13T09:17:33.818 INFO:teuthology.orchestra.run.smithi032.stdout:466 punch from 0x105cedb to 0x1060bb1, (0x3cd6 bytes) 2024-09-13T09:17:33.818 INFO:teuthology.orchestra.run.smithi032.stdout:468 read 0x8e82077 thru 0x8e8f63c (0xd5c6 bytes) 2024-09-13T09:17:33.818 INFO:teuthology.orchestra.run.smithi032.stdout:469 read 0x4af1d57 thru 0x4afc420 (0xa6ca bytes) 2024-09-13T09:17:33.818 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.818 INFO:teuthology.orchestra.run.smithi032.stdout:474 punch from 0x85ed87e to 0x85f3e99, (0x661b bytes) 2024-09-13T09:17:33.818 INFO:teuthology.orchestra.run.smithi032.stdout:476 read 0x957b4a6 thru 0x95868d3 (0xb42e bytes) 2024-09-13T09:17:33.818 INFO:teuthology.orchestra.run.smithi032.stdout:477 write 0x628393f thru 0x628d025 (0x96e7 bytes) 2024-09-13T09:17:33.824 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.824 INFO:teuthology.orchestra.run.smithi032.stdout:479 read 0x8378222 thru 0x8382197 (0x9f76 bytes) 2024-09-13T09:17:33.824 INFO:teuthology.orchestra.run.smithi032.stdout:480 write 0x330db09 thru 0x330e92c (0xe24 bytes) 2024-09-13T09:17:33.826 INFO:teuthology.orchestra.run.smithi032.stdout:483 write 0x7249fc7 thru 0x724ae4d (0xe87 bytes) 2024-09-13T09:17:33.832 INFO:teuthology.orchestra.run.smithi032.stdout:484 read 0x5dbf576 thru 0x5dc97ee (0xa279 bytes) 2024-09-13T09:17:33.832 INFO:teuthology.orchestra.run.smithi032.stdout:485 writesame 0x7207a20 thru 0x720b255 data_size 0x5(0x3836 bytes) 2024-09-13T09:17:33.836 INFO:teuthology.orchestra.run.smithi032.stdout:486 writesame 0x1628c9a thru 0x162e4e3 data_size 0x3(0x584a bytes) 2024-09-13T09:17:33.872 INFO:teuthology.orchestra.run.smithi032.stdout:488 trunc from 0xc700c5f to 0xc15a4d 2024-09-13T09:17:33.879 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:33.874+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:33.900 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:33.923 INFO:teuthology.orchestra.run.smithi032.stdout:490 write 0x4793d5e thru 0x4794789 (0xa2c bytes) 2024-09-13T09:17:33.929 INFO:teuthology.orchestra.run.smithi032.stdout:491 read 0x920034 thru 0x92c934 (0xc901 bytes) 2024-09-13T09:17:33.930 INFO:teuthology.orchestra.run.smithi032.stdout:494 punch from 0x148127f to 0x1488dda, (0x7b5b bytes) 2024-09-13T09:17:33.944 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:33.944 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:17:33.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:17:33.944 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:17:33.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.944 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:33.944 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:33.944 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.546745923, 2024-09-13T09:17:33.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077478195999999999, 2024-09-13T09:17:33.944 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392636+0000", 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392651+0000", 2024-09-13T09:17:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392674+0000", 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.264e-05 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392691+0000", 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7203000000000001e-05 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:33.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392780+0000", 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8671999999999994e-05 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392794+0000", 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.466664+0000", 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073869913999999995 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467139+0000", 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047488599999999999 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467664+0000", 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052530600000000002 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469125+0000", 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001460432 2024-09-13T09:17:33.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469847+0000", 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072197799999999996 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470006+0000", 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015955900000000001 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:33.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470116+0000", 2024-09-13T09:17:33.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109836 2024-09-13T09:17:33.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.951 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:33.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.951 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:33.951 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531091+0000", 2024-09-13T09:17:33.951 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.408292951, 2024-09-13T09:17:33.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010986811000000001, 2024-09-13T09:17:33.952 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:33.952 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:33.952 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:33.952 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:33.952 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:33.952 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 34 2024-09-13T09:17:33.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531091+0000", 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531091+0000", 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531090+0000", 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531096+0000", 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.806e-06 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531108+0000", 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.224e-05 2024-09-13T09:17:33.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531117+0000", 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6030000000000005e-06 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.536352+0000", 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052353260000000002 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.536628+0000", 2024-09-13T09:17:33.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027556500000000002 2024-09-13T09:17:33.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:33.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.536981+0000", 2024-09-13T09:17:33.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035341700000000002 2024-09-13T09:17:33.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:33.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.541700+0000", 2024-09-13T09:17:33.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047184100000000001 2024-09-13T09:17:33.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:33.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.541795+0000", 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5031999999999994e-05 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.541955+0000", 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160056 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542078+0000", 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012296200000000001 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531127+0000", 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.408257025000001, 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011389865000000001, 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 35 2024-09-13T09:17:33.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531127+0000", 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531127+0000", 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531126+0000", 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531274+0000", 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014800800000000001 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531288+0000", 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3867e-05 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531297+0000", 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9330000000000007e-06 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.538727+0000", 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.007429639 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.538949+0000", 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022249499999999999 2024-09-13T09:17:33.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.539160+0000", 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000211079 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542252+0000", 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030918460000000001 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:33.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542280+0000", 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7433e-05 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542409+0000", 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012945599999999999 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542517+0000", 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107725 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.964 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531307+0000", 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.408076549, 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012285895, 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 36 2024-09-13T09:17:33.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531307+0000", 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531307+0000", 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531306+0000", 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531313+0000", 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3130000000000004e-06 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531322+0000", 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.85e-06 2024-09-13T09:17:33.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:33.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531329+0000", 2024-09-13T09:17:33.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9600000000000003e-06 2024-09-13T09:17:33.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:33.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.540210+0000", 2024-09-13T09:17:33.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088810439999999994 2024-09-13T09:17:33.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.540477+0000", 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000267438 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.540862+0000", 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038524400000000003 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543383+0000", 2024-09-13T09:17:33.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002520764 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543447+0000", 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4063e-05 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543538+0000", 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1268999999999996e-05 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:33.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543593+0000", 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000002e-05 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531337+0000", 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.408047039, 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013657717, 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:33.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531337+0000", 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:33.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531337+0000", 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531336+0000", 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531344+0000", 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5599999999999996e-06 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:33.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531353+0000", 2024-09-13T09:17:33.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8869999999999998e-06 2024-09-13T09:17:33.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:33.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531360+0000", 2024-09-13T09:17:33.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9100000000000005e-06 2024-09-13T09:17:33.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:33.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542600+0000", 2024-09-13T09:17:33.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011239638999999999 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542747+0000", 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014643499999999999 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542947+0000", 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000200149 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543918+0000", 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00097078599999999996 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.544760+0000", 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00084245000000000001 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.544898+0000", 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013783900000000001 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.544995+0000", 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6766000000000002e-05 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.631401+0000", 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.307982542000001, 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010376106, 2024-09-13T09:17:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:33.978 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:33.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:33.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:33.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:33.978 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 86 2024-09-13T09:17:33.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.978 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:33.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:33.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631401+0000", 2024-09-13T09:17:33.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631401+0000", 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631400+0000", 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:33.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631411+0000", 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1029999999999999e-05 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631429+0000", 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7907e-05 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631443+0000", 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4283e-05 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.636777+0000", 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053336190000000004 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.636951+0000", 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174325 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.637168+0000", 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021682200000000001 2024-09-13T09:17:33.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641519+0000", 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0043510759999999997 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641638+0000", 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118573 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:33.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641721+0000", 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2803e-05 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641777+0000", 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6771999999999998e-05 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.631461+0000", 2024-09-13T09:17:33.983 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.307923069000001, 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010750297000000001, 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 87 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631461+0000", 2024-09-13T09:17:33.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:33.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631461+0000", 2024-09-13T09:17:33.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:33.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631460+0000", 2024-09-13T09:17:33.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631618+0000", 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015849899999999999 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631641+0000", 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2912999999999999e-05 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:33.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631657+0000", 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6305999999999999e-05 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.638673+0000", 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0070156300000000001 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.638802+0000", 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129049 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.639028+0000", 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022574100000000001 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641921+0000", 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002893648 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641947+0000", 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5307e-05 2024-09-13T09:17:33.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:33.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642120+0000", 2024-09-13T09:17:33.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017344499999999999 2024-09-13T09:17:33.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:33.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642211+0000", 2024-09-13T09:17:33.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0876000000000002e-05 2024-09-13T09:17:33.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.989 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.631676+0000", 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.307707634, 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011331733, 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 88 2024-09-13T09:17:33.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631676+0000", 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631676+0000", 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631675+0000", 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631687+0000", 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2163e-05 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631705+0000", 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7876999999999998e-05 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631722+0000", 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.685e-05 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.640114+0000", 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083919979999999995 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.640389+0000", 2024-09-13T09:17:33.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027471800000000002 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.640788+0000", 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00039957000000000002 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642474+0000", 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001686014 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:33.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642883+0000", 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040869700000000002 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642957+0000", 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4239000000000005e-05 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.643008+0000", 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0917000000000001e-05 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:33.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:33.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.996 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:33.996 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.797962+0000", 2024-09-13T09:17:33.996 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.141421660999999, 2024-09-13T09:17:33.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01079161, 2024-09-13T09:17:33.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:33.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:33.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:33.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:33.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:33.997 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 110 2024-09-13T09:17:33.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797962+0000", 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797962+0000", 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797960+0000", 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797974+0000", 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4545999999999999e-05 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797999+0000", 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4573e-05 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:33.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798017+0000", 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7940000000000001e-05 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.804966+0000", 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069496469999999998 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.805091+0000", 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012414199999999999 2024-09-13T09:17:34.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.805276+0000", 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018536799999999999 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808455+0000", 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00317914 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:34.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808580+0000", 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012457500000000001 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808654+0000", 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4373000000000001e-05 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808754+0000", 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9869000000000002e-05 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798038+0000", 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.141345824999998, 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012627325999999999, 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 111 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.003 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798038+0000", 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798038+0000", 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:34.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798037+0000", 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798048+0000", 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1194e-05 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798074+0000", 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5516000000000001e-05 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798088+0000", 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4663e-05 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.806212+0000", 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081233810000000007 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.806328+0000", 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011605200000000001 2024-09-13T09:17:34.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.806514+0000", 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018678299999999999 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808565+0000", 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020504439999999998 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810494+0000", 2024-09-13T09:17:34.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019292109999999999 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810569+0000", 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4436000000000004e-05 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810665+0000", 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6842999999999994e-05 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.008 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798108+0000", 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.141276199, 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014039507999999999, 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.009 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798108+0000", 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798108+0000", 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:34.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798107+0000", 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798267+0000", 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160116 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798294+0000", 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6923e-05 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798317+0000", 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3693000000000002e-05 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.807303+0000", 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089853680000000005 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.807428+0000", 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012510200000000001 2024-09-13T09:17:34.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.807616+0000", 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188145 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810479+0000", 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028628949999999998 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811969+0000", 2024-09-13T09:17:34.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001489661 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812047+0000", 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8296999999999999e-05 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812147+0000", 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100375 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798339+0000", 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.141044601000001, 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013996181999999999, 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:34.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798339+0000", 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798339+0000", 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798338+0000", 2024-09-13T09:17:34.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798357+0000", 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.897e-05 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798404+0000", 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7119000000000001e-05 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798421+0000", 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7187e-05 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808797+0000", 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010375880000000001 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808915+0000", 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011846200000000001 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809099+0000", 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183313 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811848+0000", 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027493439999999999 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812174+0000", 2024-09-13T09:17:34.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032562200000000002 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812244+0000", 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0902000000000007e-05 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812335+0000", 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0995999999999994e-05 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798442+0000", 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.140941841, 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015723348000000002, 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.021 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798442+0000", 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798442+0000", 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798441+0000", 2024-09-13T09:17:34.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798453+0000", 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2784e-05 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798476+0000", 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2875999999999999e-05 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798490+0000", 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3482999999999999e-05 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809646+0000", 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011155847 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809754+0000", 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108256 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809965+0000", 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000210659 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811953+0000", 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019889439999999999 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813962+0000", 2024-09-13T09:17:34.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020080850000000002 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814040+0000", 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8184999999999998e-05 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814165+0000", 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012563299999999999 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798506+0000", 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.140877472, 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015834198000000001, 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 115 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.027 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798506+0000", 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798506+0000", 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798505+0000", 2024-09-13T09:17:34.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798658+0000", 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000153005 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798681+0000", 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2634e-05 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798697+0000", 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5696e-05 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810746+0000", 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012049331 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810873+0000", 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012754899999999999 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811051+0000", 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000177252 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813948+0000", 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028975239999999998 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814188+0000", 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000239724 2024-09-13T09:17:34.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:34.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814255+0000", 2024-09-13T09:17:34.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7372999999999993e-05 2024-09-13T09:17:34.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:34.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814341+0000", 2024-09-13T09:17:34.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5270000000000002e-05 2024-09-13T09:17:34.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.032 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798716+0000", 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.140667626999999, 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015827386999999998, 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116 2024-09-13T09:17:34.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798716+0000", 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798716+0000", 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:34.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798715+0000", 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798727+0000", 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1800000000000001e-05 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798746+0000", 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8992999999999999e-05 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798765+0000", 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9049e-05 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812383+0000", 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013618531999999999 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812498+0000", 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115056 2024-09-13T09:17:34.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812682+0000", 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018366899999999999 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814367+0000", 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00168501 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:34.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814421+0000", 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4199000000000001e-05 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814492+0000", 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0502999999999999e-05 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814544+0000", 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2039000000000001e-05 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798781+0000", 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.140602740999999, 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016163997999999999, 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 117 2024-09-13T09:17:34.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798781+0000", 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798781+0000", 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798780+0000", 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798795+0000", 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4574e-05 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798813+0000", 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8536e-05 2024-09-13T09:17:34.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798829+0000", 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5866999999999999e-05 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813236+0000", 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014407229000000001 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813350+0000", 2024-09-13T09:17:34.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011376500000000001 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813530+0000", 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179619 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814654+0000", 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011244339999999999 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:34.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814768+0000", 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011397599999999999 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814845+0000", 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7299999999999995e-05 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814945+0000", 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9875000000000002e-05 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.044 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:12.288452+0000", 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.650932055999998, 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010418200000000001, 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140 2024-09-13T09:17:34.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288452+0000", 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288452+0000", 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288451+0000", 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288508+0000", 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6706999999999998e-05 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288517+0000", 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8339999999999994e-06 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288523+0000", 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5419999999999996e-06 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.291720+0000", 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031973499999999998 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.291765+0000", 2024-09-13T09:17:34.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.443e-05 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.291834+0000", 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9053000000000004e-05 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.293879+0000", 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020453619999999998 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.298804+0000", 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049248069999999998 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.298831+0000", 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7436e-05 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.298870+0000", 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8549e-05 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.050 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:12.288530+0000", 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.650854099, 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011049003999999999, 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:34.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 141 2024-09-13T09:17:34.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.052 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:34.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:34.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288530+0000", 2024-09-13T09:17:34.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:34.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288530+0000", 2024-09-13T09:17:34.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288529+0000", 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288532+0000", 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:34.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288536+0000", 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8480000000000003e-06 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288539+0000", 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0479999999999999e-06 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.292188+0000", 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036491639999999999 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.292357+0000", 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016818200000000001 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.292492+0000", 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013562099999999999 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.293885+0000", 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001393044 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:34.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.299519+0000", 2024-09-13T09:17:34.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056339670000000001 2024-09-13T09:17:34.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:34.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.299547+0000", 2024-09-13T09:17:34.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7928000000000001e-05 2024-09-13T09:17:34.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:34.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.299579+0000", 2024-09-13T09:17:34.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1547999999999997e-05 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.899283741000001, 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010963133, 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:34.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:34.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:34.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 158 2024-09-13T09:17:34.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.058 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:34.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:34.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:34.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:34.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040255+0000", 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015546599999999999 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040268+0000", 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2860000000000001e-05 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040276+0000", 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5399999999999996e-06 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.047322+0000", 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0070454380000000002 2024-09-13T09:17:34.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.047443+0000", 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012126 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.047631+0000", 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000187558 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:34.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050775+0000", 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031440000000000001 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050797+0000", 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2730000000000001e-05 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050928+0000", 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013016500000000001 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051063+0000", 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013571300000000001 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:13.040286+0000", 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.899097578999999, 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011356279, 2024-09-13T09:17:34.063 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 159 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040286+0000", 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040286+0000", 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040285+0000", 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040296+0000", 2024-09-13T09:17:34.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0473e-05 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040305+0000", 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0070000000000003e-06 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040315+0000", 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8500000000000006e-06 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:34.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.049739+0000", 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094246899999999995 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.049892+0000", 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015284900000000001 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050097+0000", 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020424500000000001 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051387+0000", 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012901970000000001 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051511+0000", 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124039 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051589+0000", 2024-09-13T09:17:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.873e-05 2024-09-13T09:17:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051643+0000", 2024-09-13T09:17:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3102000000000001e-05 2024-09-13T09:17:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:34.069 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:34.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:17:34.072 INFO:teuthology.orchestra.run.smithi032.stdout:495 writesame 0xa0cd6d6 thru 0xa0cf250 data_size 0xf(0x1b7b bytes) 2024-09-13T09:17:34.072 INFO:teuthology.orchestra.run.smithi032.stdout:496 trunc from 0xa0cf251 to 0xaec9180 2024-09-13T09:17:34.072 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:34.072 INFO:teuthology.orchestra.run.smithi032.stdout:499 write 0xa6ce204 thru 0xa6d63ff (0x81fc bytes) 2024-09-13T09:17:34.072 INFO:teuthology.orchestra.run.smithi032.stdout:500 trunc from 0xaec9180 to 0x3e062ce 2024-09-13T09:17:34.073 INFO:teuthology.orchestra.run.smithi032.stdout:502 writesame 0x882e3b6 thru 0x88374cb data_size 0x31(0x9116 bytes) 2024-09-13T09:17:34.073 INFO:teuthology.orchestra.run.smithi032.stdout:506 read 0x6e6f5a thru 0x6f583f (0xe8e6 bytes) 2024-09-13T09:17:34.073 INFO:teuthology.orchestra.run.smithi032.stdout:508 read 0x127ed91 thru 0x128c0b7 (0xd327 bytes) 2024-09-13T09:17:34.073 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:34.073 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:34.068+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:34.079 INFO:teuthology.orchestra.run.smithi032.stdout:511 writesame 0xd7e8701 thru 0xd7f48f1 data_size 0x1(0xc1f1 bytes) 2024-09-13T09:17:34.086 INFO:teuthology.orchestra.run.smithi032.stdout:512 trunc from 0xd7f48f2 to 0xabd20b9 2024-09-13T09:17:34.111 INFO:teuthology.orchestra.run.smithi032.stdout:513 writesame 0xabec5c8 thru 0xabfa0fe data_size 0x1(0xdb37 bytes) 2024-09-13T09:17:34.131 INFO:teuthology.orchestra.run.smithi032.stdout:514 writesame 0xdcd0f4e thru 0xdcd9edd data_size 0x8(0x8f90 bytes) 2024-09-13T09:17:34.131 INFO:teuthology.orchestra.run.smithi032.stdout:516 punch from 0xd961783 to 0xd96e4c2, (0xcd3f bytes) 2024-09-13T09:17:34.131 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:34.131 INFO:teuthology.orchestra.run.smithi032.stdout:522 read 0x4f0eb2f thru 0x4f1b0cf (0xc5a1 bytes) 2024-09-13T09:17:34.154 INFO:teuthology.orchestra.run.smithi032.stdout:523 trunc from 0xdcd9ede to 0x938e24 2024-09-13T09:17:34.172 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:34.168+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:34.215 INFO:teuthology.orchestra.run.smithi032.stdout:524 write 0xc3b01bc thru 0xc3b71dd (0x7022 bytes) 2024-09-13T09:17:34.222 INFO:teuthology.orchestra.run.smithi032.stdout:525 punch from 0x4cff622 to 0x4d0f422, (0xfe00 bytes) 2024-09-13T09:17:34.222 INFO:teuthology.orchestra.run.smithi032.stdout:526 read 0xa91502f thru 0xa921243 (0xc215 bytes) 2024-09-13T09:17:34.239 INFO:teuthology.orchestra.run.smithi032.stdout:528 trunc from 0xc3b71de to 0x27de0de 2024-09-13T09:17:34.247 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:34.248 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:34.263 INFO:teuthology.orchestra.run.smithi032.stdout:529 punch from 0x22a5fa8 to 0x22b2a1f, (0xca77 bytes) 2024-09-13T09:17:34.263 INFO:teuthology.orchestra.run.smithi032.stdout:530 read 0xdd5b8a thru 0xdd7066 (0x14dd bytes) 2024-09-13T09:17:34.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:17:34.266 INFO:teuthology.orchestra.run.smithi032.stdout:532 trunc from 0x27de0de to 0x38306d3 2024-09-13T09:17:34.273 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:34.266+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:34.299 INFO:teuthology.orchestra.run.smithi032.stdout:533 writesame 0x900f0d0 thru 0x9013d81 data_size 0x1(0x4cb2 bytes) 2024-09-13T09:17:34.299 INFO:teuthology.orchestra.run.smithi032.stdout:534 writesame 0x503958 thru 0x513894 data_size 0x13(0xff3d bytes) 2024-09-13T09:17:34.305 INFO:teuthology.orchestra.run.smithi032.stdout:535 write 0x1ddaf74 thru 0x1ddbaa5 (0xb32 bytes) 2024-09-13T09:17:34.310 INFO:teuthology.orchestra.run.smithi032.stdout:537 read 0x5eb9185 thru 0x5ebad02 (0x1b7e bytes) 2024-09-13T09:17:34.319 INFO:teuthology.orchestra.run.smithi032.stdout:540 trunc from 0x9013d82 to 0xe910ef5 2024-09-13T09:17:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:541 write 0x240c918 thru 0x24154d6 (0x8bbf bytes) 2024-09-13T09:17:34.339 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:34.339 INFO:teuthology.orchestra.run.smithi032.stdout:544 read 0x3035f03 thru 0x30379e5 (0x1ae3 bytes) 2024-09-13T09:17:34.339 INFO:teuthology.orchestra.run.smithi032.stdout:545 read 0xcb90fe0 thru 0xcb9feef (0xef10 bytes) 2024-09-13T09:17:34.339 INFO:teuthology.orchestra.run.smithi032.stdout:546 write 0x1c5a208 thru 0x1c60c41 (0x6a3a bytes) 2024-09-13T09:17:34.345 INFO:teuthology.orchestra.run.smithi032.stdout:547 writesame 0x8628001 thru 0x862fb40 data_size 0x74(0x7b40 bytes) 2024-09-13T09:17:34.363 INFO:teuthology.orchestra.run.smithi032.stdout:548 trunc from 0xe910ef5 to 0x77c2609 2024-09-13T09:17:34.374 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:34.372+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:34.379 INFO:teuthology.orchestra.run.smithi032.stdout:549 punch from 0x73d92ba to 0x73e59fa, (0xc740 bytes) 2024-09-13T09:17:34.387 INFO:teuthology.orchestra.run.smithi032.stdout:550 trunc from 0x77c2609 to 0xcca417d 2024-09-13T09:17:34.399 INFO:teuthology.orchestra.run.smithi032.stdout:555 write 0x9804535 thru 0x980606b (0x1b37 bytes) 2024-09-13T09:17:34.402 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:34.403 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:34.408 INFO:teuthology.orchestra.run.smithi032.stdout:556 punch from 0x89e4106 to 0x89e4ffe, (0xef8 bytes) 2024-09-13T09:17:34.409 INFO:teuthology.orchestra.run.smithi032.stdout:557 punch from 0xc80bcde to 0xc81a80f, (0xeb31 bytes) 2024-09-13T09:17:34.417 INFO:teuthology.orchestra.run.smithi032.stdout:560 trunc from 0xcca417d to 0x74e557c 2024-09-13T09:17:34.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:17:34.450 INFO:teuthology.orchestra.run.smithi032.stdout:561 trunc from 0x74e557c to 0x420febc 2024-09-13T09:17:34.470 INFO:teuthology.orchestra.run.smithi032.stdout:564 read 0x3e8c009 thru 0x3e99258 (0xd250 bytes) 2024-09-13T09:17:34.475 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:34.472+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:34.490 INFO:teuthology.orchestra.run.smithi032.stdout:565 write 0x7f9028f thru 0x7f93977 (0x36e9 bytes) 2024-09-13T09:17:34.520 INFO:teuthology.orchestra.run.smithi032.stdout:566 writesame 0xc22f297 thru 0xc2390d6 data_size 0x20(0x9e40 bytes) 2024-09-13T09:17:34.520 INFO:teuthology.orchestra.run.smithi032.stdout:567 write 0x9766d99 thru 0x976b4c0 (0x4728 bytes) 2024-09-13T09:17:34.527 INFO:teuthology.orchestra.run.smithi032.stdout:569 read 0x5c0a131 thru 0x5c169a4 (0xc874 bytes) 2024-09-13T09:17:34.540 INFO:teuthology.orchestra.run.smithi032.stdout:570 trunc from 0xc2390d7 to 0x5fd3f11 2024-09-13T09:17:34.565 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:34.576 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:34.572+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:34.587 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-13T09:17:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:11.579976+0000", 2024-09-13T09:17:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.005182982000001, 2024-09-13T09:17:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002347348, 2024-09-13T09:17:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.579976+0000", 2024-09-13T09:17:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.579976+0000", 2024-09-13T09:17:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.579974+0000", 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.579994+0000", 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0083000000000001e-05 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.580032+0000", 2024-09-13T09:17:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7741999999999999e-05 2024-09-13T09:17:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.580069+0000", 2024-09-13T09:17:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7177e-05 2024-09-13T09:17:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.580185+0000", 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011610199999999999 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.580223+0000", 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7540000000000003e-05 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.582239+0000", 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020165679999999998 2024-09-13T09:17:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.582324+0000", 2024-09-13T09:17:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4369000000000005e-05 2024-09-13T09:17:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:57 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-13T09:17:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:11.586713+0000", 2024-09-13T09:17:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.998445794999999, 2024-09-13T09:17:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0023464390000000001, 2024-09-13T09:17:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586713+0000", 2024-09-13T09:17:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586713+0000", 2024-09-13T09:17:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586712+0000", 2024-09-13T09:17:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586767+0000", 2024-09-13T09:17:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5726999999999997e-05 2024-09-13T09:17:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586797+0000", 2024-09-13T09:17:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9266999999999999e-05 2024-09-13T09:17:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586811+0000", 2024-09-13T09:17:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4679e-05 2024-09-13T09:17:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586916+0000", 2024-09-13T09:17:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010517 2024-09-13T09:17:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586952+0000", 2024-09-13T09:17:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5228999999999998e-05 2024-09-13T09:17:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.588980+0000", 2024-09-13T09:17:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020282009999999999 2024-09-13T09:17:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.589060+0000", 2024-09-13T09:17:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0023000000000003e-05 2024-09-13T09:17:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:11.608699+0000", 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.976460311, 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025443969999999999, 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608699+0000", 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608699+0000", 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608697+0000", 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608757+0000", 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9552999999999997e-05 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608793+0000", 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5902999999999998e-05 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608812+0000", 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9233e-05 2024-09-13T09:17:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608987+0000", 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017490600000000001 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.609022+0000", 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4946000000000001e-05 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.611165+0000", 2024-09-13T09:17:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021434539999999999 2024-09-13T09:17:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.611243+0000", 2024-09-13T09:17:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8242000000000001e-05 2024-09-13T09:17:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:129 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'87, mlcod=16'85)", 2024-09-13T09:17:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.288944+0000", 2024-09-13T09:17:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.296215474, 2024-09-13T09:17:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0048963449999999999, 2024-09-13T09:17:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288944+0000", 2024-09-13T09:17:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288944+0000", 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288942+0000", 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288960+0000", 2024-09-13T09:17:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8077e-05 2024-09-13T09:17:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288989+0000", 2024-09-13T09:17:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9476e-05 2024-09-13T09:17:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289003+0000", 2024-09-13T09:17:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3046e-05 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289103+0000", 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010054 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289137+0000", 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3575999999999998e-05 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293750+0000", 2024-09-13T09:17:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046137610000000001 2024-09-13T09:17:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293840+0000", 2024-09-13T09:17:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9698999999999996e-05 2024-09-13T09:17:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:130 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'88, mlcod=16'85)", 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.289290+0000", 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.295869369999998, 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046447700000000003, 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289290+0000", 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289290+0000", 2024-09-13T09:17:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289289+0000", 2024-09-13T09:17:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289299+0000", 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6430000000000007e-06 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289315+0000", 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.609e-05 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289324+0000", 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3430000000000002e-06 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289718+0000", 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039453699999999998 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289792+0000", 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3786000000000001e-05 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293848+0000", 2024-09-13T09:17:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0040556020000000002 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293935+0000", 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6749000000000006e-05 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:131 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.289777+0000", 2024-09-13T09:17:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.295382521000001, 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005922531, 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289777+0000", 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289777+0000", 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289775+0000", 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289961+0000", 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018587200000000001 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289993+0000", 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2007e-05 2024-09-13T09:17:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290016+0000", 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3025999999999999e-05 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290280+0000", 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026384800000000001 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290318+0000", 2024-09-13T09:17:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7945999999999999e-05 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295642+0000", 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0053246159999999999 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295699+0000", 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6898999999999999e-05 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:132 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.290371+0000", 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.294788071999999, 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0053750650000000001, 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290371+0000", 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290371+0000", 2024-09-13T09:17:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290369+0000", 2024-09-13T09:17:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290384+0000", 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5133999999999999e-05 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290419+0000", 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4555999999999999e-05 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290437+0000", 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8343e-05 2024-09-13T09:17:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291376+0000", 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00093908299999999995 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291410+0000", 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3883000000000001e-05 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295705+0000", 2024-09-13T09:17:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042944790000000004 2024-09-13T09:17:34.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295746+0000", 2024-09-13T09:17:34.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1496999999999999e-05 2024-09-13T09:17:34.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.621 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.621 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:133 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-13T09:17:34.621 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.290748+0000", 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.294411051000001, 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0050401270000000001, 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290748+0000", 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290748+0000", 2024-09-13T09:17:34.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290747+0000", 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290761+0000", 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4236000000000001e-05 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290788+0000", 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6826999999999999e-05 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290804+0000", 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5798999999999999e-05 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291981+0000", 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011771080000000001 2024-09-13T09:17:34.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292015+0000", 2024-09-13T09:17:34.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4693000000000001e-05 2024-09-13T09:17:34.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295750+0000", 2024-09-13T09:17:34.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037345780000000001 2024-09-13T09:17:34.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295788+0000", 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:134 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.291161+0000", 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.293998325, 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064130539999999996, 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291161+0000", 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291161+0000", 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291159+0000", 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291257+0000", 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7498999999999996e-05 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291284+0000", 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291305+0000", 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1773e-05 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292753+0000", 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014473719999999999 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292786+0000", 2024-09-13T09:17:34.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3370000000000001e-05 2024-09-13T09:17:34.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297518+0000", 2024-09-13T09:17:34.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0047318569999999999 2024-09-13T09:17:34.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297574+0000", 2024-09-13T09:17:34.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6026000000000002e-05 2024-09-13T09:17:34.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.630 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:135 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'85)", 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.291641+0000", 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.293518225, 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0059934710000000002, 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291641+0000", 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291641+0000", 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291639+0000", 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291653+0000", 2024-09-13T09:17:34.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3977e-05 2024-09-13T09:17:34.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291684+0000", 2024-09-13T09:17:34.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0298999999999999e-05 2024-09-13T09:17:34.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291701+0000", 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7847e-05 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294015+0000", 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002313402 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294047+0000", 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2063000000000001e-05 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297582+0000", 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035351430000000001 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297634+0000", 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2460000000000003e-05 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.635 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'85)", 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.292054+0000", 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.293104850999999, 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0073146829999999998, 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292054+0000", 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292054+0000", 2024-09-13T09:17:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292053+0000", 2024-09-13T09:17:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292067+0000", 2024-09-13T09:17:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.411e-05 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292092+0000", 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5173000000000001e-05 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292111+0000", 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8963000000000001e-05 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294704+0000", 2024-09-13T09:17:34.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025933800000000002 2024-09-13T09:17:34.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294735+0000", 2024-09-13T09:17:34.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0366e-05 2024-09-13T09:17:34.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299316+0000", 2024-09-13T09:17:34.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004581412 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299369+0000", 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3016000000000002e-05 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:137 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'85)", 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.292515+0000", 2024-09-13T09:17:34.640 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.292644641999999, 2024-09-13T09:17:34.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068905700000000004, 2024-09-13T09:17:34.641 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.641 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.641 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292515+0000", 2024-09-13T09:17:34.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292515+0000", 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292513+0000", 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292586+0000", 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3139999999999994e-05 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292618+0000", 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1878999999999998e-05 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292639+0000", 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0752999999999999e-05 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295880+0000", 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0032411889999999998 2024-09-13T09:17:34.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295927+0000", 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7175999999999997e-05 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299375+0000", 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0034476400000000001 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299405+0000", 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0642999999999998e-05 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:138 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'85)", 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.292944+0000", 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.292215007999999, 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064997630000000004, 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292944+0000", 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292944+0000", 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292943+0000", 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:34.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292957+0000", 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4610000000000001e-05 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292982+0000", 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5003000000000001e-05 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293000+0000", 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8142999999999999e-05 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.296939+0000", 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0039383700000000001 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.296972+0000", 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2716999999999999e-05 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299412+0000", 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024409340000000001 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299444+0000", 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.154e-05 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:139 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'85)", 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.293238+0000", 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.291921124000002, 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062557749999999999, 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293238+0000", 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293238+0000", 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293237+0000", 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293250+0000", 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.379e-05 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293276+0000", 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6126e-05 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293295+0000", 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8904000000000001e-05 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297711+0000", 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044153789999999997 2024-09-13T09:17:34.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297745+0000", 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4499999999999998e-05 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299450+0000", 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00170436 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299494+0000", 2024-09-13T09:17:34.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4302999999999999e-05 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'85)", 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.293606+0000", 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.291552846999998, 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066098429999999998, 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293606+0000", 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293606+0000", 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293605+0000", 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293707+0000", 2024-09-13T09:17:34.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010204600000000001 2024-09-13T09:17:34.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293732+0000", 2024-09-13T09:17:34.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5429999999999999e-05 2024-09-13T09:17:34.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293749+0000", 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6390000000000001e-05 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.298325+0000", 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045769440000000003 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.298370+0000", 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4187000000000002e-05 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300157+0000", 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017874690000000001 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300216+0000", 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9067000000000002e-05 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:141 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'85)", 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.294217+0000", 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.290942611999998, 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067059559999999999, 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294217+0000", 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294217+0000", 2024-09-13T09:17:34.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294215+0000", 2024-09-13T09:17:34.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:34.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294229+0000", 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3974e-05 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294261+0000", 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1903000000000003e-05 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294280+0000", 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9589000000000002e-05 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299531+0000", 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0052504029999999998 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299567+0000", 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6427000000000002e-05 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300868+0000", 2024-09-13T09:17:34.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013010470000000001 2024-09-13T09:17:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300923+0000", 2024-09-13T09:17:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4203000000000003e-05 2024-09-13T09:17:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:142 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'100, mlcod=16'98)", 2024-09-13T09:17:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.304015+0000", 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.281144214000001, 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024443709999999999, 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304015+0000", 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304015+0000", 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304013+0000", 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304261+0000", 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024744099999999999 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304284+0000", 2024-09-13T09:17:34.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3370000000000002e-05 2024-09-13T09:17:34.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304298+0000", 2024-09-13T09:17:34.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3502999999999999e-05 2024-09-13T09:17:34.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304384+0000", 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6650000000000006e-05 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304424+0000", 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9941999999999998e-05 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.306398+0000", 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019733390000000002 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.306459+0000", 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1869e-05 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:143 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.304318+0000", 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.280841565999999, 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029677929999999998, 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.668 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304318+0000", 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304318+0000", 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304317+0000", 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304426+0000", 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109209 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304450+0000", 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4090000000000001e-05 2024-09-13T09:17:34.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304462+0000", 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2480000000000001e-05 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.305073+0000", 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00061091499999999996 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.305112+0000", 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8523999999999999e-05 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307197+0000", 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020850439999999999 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307285+0000", 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8684999999999995e-05 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.672 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.673 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:144 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-13T09:17:34.673 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.304477+0000", 2024-09-13T09:17:34.673 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.280682287000001, 2024-09-13T09:17:34.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029682900000000002, 2024-09-13T09:17:34.673 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.673 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.673 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304477+0000", 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304477+0000", 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304476+0000", 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304603+0000", 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012634500000000001 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304620+0000", 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7450000000000001e-05 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304629+0000", 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4599999999999992e-06 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.305775+0000", 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001145725 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.305808+0000", 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2932999999999998e-05 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307394+0000", 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015858269999999999 2024-09-13T09:17:34.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307445+0000", 2024-09-13T09:17:34.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1329999999999998e-05 2024-09-13T09:17:34.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.677 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.677 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:145 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-13T09:17:34.677 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.304643+0000", 2024-09-13T09:17:34.677 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.280516629000001, 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0032891209999999999, 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304643+0000", 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304643+0000", 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:34.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304642+0000", 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304743+0000", 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101049 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304775+0000", 2024-09-13T09:17:34.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1973e-05 2024-09-13T09:17:34.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:34.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304783+0000", 2024-09-13T09:17:34.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-13T09:17:34.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:34.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.306544+0000", 2024-09-13T09:17:34.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017610659999999999 2024-09-13T09:17:34.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.306577+0000", 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3149999999999999e-05 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307851+0000", 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001273843 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307932+0000", 2024-09-13T09:17:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0465999999999998e-05 2024-09-13T09:17:34.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.682 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:34.682 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:34.682 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:34.683 INFO:teuthology.orchestra.run.smithi032.stdout:573 write 0xde7b948 thru 0xde83997 (0x8050 bytes) 2024-09-13T09:17:34.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:17:34.685 INFO:teuthology.orchestra.run.smithi032.stdout:574 writesame 0x7bacfbd thru 0x7bb5f4a data_size 0x7(0x8f8e bytes) 2024-09-13T09:17:34.685 INFO:teuthology.orchestra.run.smithi032.stdout:575 trunc from 0xde83998 to 0xbae1bfb 2024-09-13T09:17:34.685 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:34.685 INFO:teuthology.orchestra.run.smithi032.stdout:579 read 0x82b62d7 thru 0x82b7ff7 (0x1d21 bytes) 2024-09-13T09:17:34.685 INFO:teuthology.orchestra.run.smithi032.stdout:580 writesame 0xa23c6ea thru 0xa24308a data_size 0x1(0x69a1 bytes) 2024-09-13T09:17:34.685 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:34.680+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:34.688 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:34.688 INFO:teuthology.orchestra.run.smithi032.stdout:582 writesame 0x6387ebe thru 0x638d395 data_size 0x3c(0x54d8 bytes) 2024-09-13T09:17:34.695 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:34.695 INFO:teuthology.orchestra.run.smithi032.stdout:584 writesame 0xb0946d9 thru 0xb09a01a data_size 0x19(0x5942 bytes) 2024-09-13T09:17:34.702 INFO:teuthology.orchestra.run.smithi032.stdout:585 read 0x1784bca thru 0x17938c5 (0xecfc bytes) 2024-09-13T09:17:34.704 INFO:teuthology.orchestra.run.smithi032.stdout:586 write 0x53d54fe thru 0x53db568 (0x606b bytes) 2024-09-13T09:17:34.712 INFO:teuthology.orchestra.run.smithi032.stdout:588 punch from 0x7642763 to 0x7643039, (0x8d6 bytes) 2024-09-13T09:17:34.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:17:34.741 INFO:teuthology.orchestra.run.smithi032.stdout:590 trunc from 0xbae1bfb to 0x181d118 2024-09-13T09:17:34.784 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:34.780+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:34.785 INFO:teuthology.orchestra.run.smithi032.stdout:591 trunc from 0x181d118 to 0xc6f5a47 2024-09-13T09:17:34.796 INFO:teuthology.orchestra.run.smithi032.stdout:592 write 0x26cfd6 thru 0x27430f (0x733a bytes) 2024-09-13T09:17:34.798 INFO:teuthology.orchestra.run.smithi032.stdout:593 read 0x7dd1e61 thru 0x7ddf753 (0xd8f3 bytes) 2024-09-13T09:17:34.798 INFO:teuthology.orchestra.run.smithi032.stdout:594 write 0x1983807 thru 0x198452f (0xd29 bytes) 2024-09-13T09:17:34.802 INFO:teuthology.orchestra.run.smithi032.stdout:595 read 0x4ffa913 thru 0x4ffca7a (0x2168 bytes) 2024-09-13T09:17:34.818 INFO:teuthology.orchestra.run.smithi032.stdout:596 writesame 0xeb0c04b thru 0xeb19640 data_size 0x66(0xd5f6 bytes) 2024-09-13T09:17:34.818 INFO:teuthology.orchestra.run.smithi032.stdout:597 clone 2 order 21 su 524288 sc 12 2024-09-13T09:17:34.832 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:34.832 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:34.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:17:34.890 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:34.888+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:34.987 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:34.984+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:35.005 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:35.006 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:35.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:17:35.089 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:35.086+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:35.098 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:17:35.182 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:35.182 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:35.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:35.183 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:35.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.183 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:995 3.1a e28/22 3:5db8ce44:::rbd_data.1185192b47c4.000000000000001e:head v 28'10, mlcod=27'8)", 2024-09-13T09:17:35.183 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.173001+0000", 2024-09-13T09:17:35.183 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 2.0083151529999999, 2024-09-13T09:17:35.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030959490000000002, 2024-09-13T09:17:35.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.183 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.173001+0000", 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.173001+0000", 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.173000+0000", 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.174300+0000", 2024-09-13T09:17:35.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001300278 2024-09-13T09:17:35.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.174313+0000", 2024-09-13T09:17:35.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.236e-05 2024-09-13T09:17:35.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.174320+0000", 2024-09-13T09:17:35.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9999999999999999e-06 2024-09-13T09:17:35.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.174422+0000", 2024-09-13T09:17:35.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101668 2024-09-13T09:17:35.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.174462+0000", 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0593000000000003e-05 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.176028+0000", 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015657749999999999 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.176097+0000", 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9091999999999995e-05 2024-09-13T09:17:35.187 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.188 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.188 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.188 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1019 3.19 3:9c410407:::rbd_data.1185192b47c4.0000000000000039:head [set-alloc-hint object_size 4194304 write_size 4194304,write 425877~24501] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:35.188 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.253485+0000", 2024-09-13T09:17:35.188 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.927831284, 2024-09-13T09:17:35.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033370040000000002, 2024-09-13T09:17:35.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.188 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.188 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.188 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.189 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.189 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1019 2024-09-13T09:17:35.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.189 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253485+0000", 2024-09-13T09:17:35.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253485+0000", 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253481+0000", 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253542+0000", 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0723000000000001e-05 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253583+0000", 2024-09-13T09:17:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0822999999999999e-05 2024-09-13T09:17:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253600+0000", 2024-09-13T09:17:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6883000000000001e-05 2024-09-13T09:17:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253712+0000", 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011202 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253993+0000", 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028126399999999998 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.254252+0000", 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025908799999999998 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.256125+0000", 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001872439 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.256673+0000", 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054806199999999998 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.256758+0000", 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5439999999999995e-05 2024-09-13T09:17:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.256822+0000", 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3859e-05 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1021 3.19 3:9c410407:::rbd_data.1185192b47c4.0000000000000039:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.274629+0000", 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.906687654, 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0026526729999999999, 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1021 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.274629+0000", 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.274629+0000", 2024-09-13T09:17:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.274625+0000", 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.274641+0000", 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.554e-05 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.274680+0000", 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9583999999999999e-05 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.274699+0000", 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8640000000000001e-05 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.274800+0000", 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100639 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.274957+0000", 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000157882 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.275131+0000", 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000173728 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.276383+0000", 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001251491 2024-09-13T09:17:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.277091+0000", 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00070886799999999998 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.277170+0000", 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8522999999999993e-05 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.277281+0000", 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111195 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1035 3.0 3:02e5ffb1:::rbd_data.1185192b47c4.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 3273017~65270] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.298987+0000", 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.882329132, 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0034987960000000002, 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1035 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.298987+0000", 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.298987+0000", 2024-09-13T09:17:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.298985+0000", 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.299088+0000", 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103199 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.299121+0000", 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2546999999999999e-05 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.299145+0000", 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4000000000000001e-05 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.299274+0000", 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012890500000000001 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.299530+0000", 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000256035 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.299845+0000", 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031564000000000001 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.302085+0000", 2024-09-13T09:17:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022398029999999998 2024-09-13T09:17:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.302341+0000", 2024-09-13T09:17:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025572799999999999 2024-09-13T09:17:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.302427+0000", 2024-09-13T09:17:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6459999999999996e-05 2024-09-13T09:17:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.302486+0000", 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8649e-05 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1078 3.4 3:2221ff39:::rbd_data.1185192b47c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 3023129~32394] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.397885+0000", 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.7834308750000001, 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035396859999999998, 2024-09-13T09:17:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1078 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.397885+0000", 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.397885+0000", 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.397882+0000", 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.397924+0000", 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2002999999999997e-05 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.397964+0000", 2024-09-13T09:17:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9829000000000001e-05 2024-09-13T09:17:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.397981+0000", 2024-09-13T09:17:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.649e-05 2024-09-13T09:17:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.398081+0000", 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100373 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.398333+0000", 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025160800000000001 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.398696+0000", 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036374699999999999 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.400471+0000", 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001775113 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.401269+0000", 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00079757700000000005 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.401357+0000", 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8518999999999997e-05 2024-09-13T09:17:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.401425+0000", 2024-09-13T09:17:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7500000000000001e-05 2024-09-13T09:17:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1090 3.6 3:6098d441:::rbd_data.1185192b47c4.0000000000000036:head [set-alloc-hint object_size 4194304 write_size 4194304,write 784719~35429] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.421825+0000", 2024-09-13T09:17:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.7594916170000001, 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003158639, 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1090 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.421825+0000", 2024-09-13T09:17:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.421825+0000", 2024-09-13T09:17:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.421822+0000", 2024-09-13T09:17:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.421862+0000", 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9956000000000002e-05 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.421895+0000", 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3869999999999999e-05 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.421911+0000", 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5356e-05 2024-09-13T09:17:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.422032+0000", 2024-09-13T09:17:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000121119 2024-09-13T09:17:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.422256+0000", 2024-09-13T09:17:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022440599999999999 2024-09-13T09:17:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.422476+0000", 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021970799999999999 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.424202+0000", 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017259129999999999 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.424833+0000", 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00063072200000000001 2024-09-13T09:17:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.424921+0000", 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8468999999999996e-05 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.424983+0000", 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2063000000000005e-05 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1123 3.0 3:0731d9f1:::rbd_data.1185192b47c4.0000000000000032:head [set-alloc-hint object_size 4194304 write_size 4194304,write 793280~26123] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.501430+0000", 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.679886204, 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0034807800000000002, 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1123 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501430+0000", 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501430+0000", 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501427+0000", 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501460+0000", 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2972000000000002e-05 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501492+0000", 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1937000000000003e-05 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501510+0000", 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7733000000000001e-05 2024-09-13T09:17:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501609+0000", 2024-09-13T09:17:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9445999999999996e-05 2024-09-13T09:17:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501844+0000", 2024-09-13T09:17:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023497500000000001 2024-09-13T09:17:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.502071+0000", 2024-09-13T09:17:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022678499999999999 2024-09-13T09:17:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.503823+0000", 2024-09-13T09:17:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017522690000000001 2024-09-13T09:17:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.504689+0000", 2024-09-13T09:17:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00086600700000000002 2024-09-13T09:17:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.504848+0000", 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000158632 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.504911+0000", 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2626999999999995e-05 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1126 3.0 3:02e5ffb1:::rbd_data.1185192b47c4.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.543493+0000", 2024-09-13T09:17:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.637823289, 2024-09-13T09:17:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029711500000000001, 2024-09-13T09:17:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1126 2024-09-13T09:17:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543493+0000", 2024-09-13T09:17:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543493+0000", 2024-09-13T09:17:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543490+0000", 2024-09-13T09:17:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543507+0000", 2024-09-13T09:17:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6773e-05 2024-09-13T09:17:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543558+0000", 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1069999999999997e-05 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543576+0000", 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8850000000000001e-05 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543670+0000", 2024-09-13T09:17:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3103000000000006e-05 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543813+0000", 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000143862 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.544001+0000", 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187608 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.545551+0000", 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001550095 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.546305+0000", 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075378700000000001 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.546406+0000", 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101523 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.546464+0000", 2024-09-13T09:17:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7618999999999998e-05 2024-09-13T09:17:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1137 3.0 3:02cec37e:::rbd_data.1185192b47c4.0000000000000019:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.547808+0000", 2024-09-13T09:17:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.6335083130000001, 2024-09-13T09:17:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003778712, 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1137 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.547808+0000", 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.547808+0000", 2024-09-13T09:17:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.547805+0000", 2024-09-13T09:17:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.547821+0000", 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6093e-05 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.547881+0000", 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9256000000000003e-05 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.547905+0000", 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4057e-05 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.548025+0000", 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120442 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.548260+0000", 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023489100000000001 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.548534+0000", 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027368500000000002 2024-09-13T09:17:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551296+0000", 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027617459999999998 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551386+0000", 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0031999999999995e-05 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551498+0000", 2024-09-13T09:17:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112716 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551587+0000", 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8313000000000006e-05 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1145 3.0 3:0731d9f1:::rbd_data.1185192b47c4.0000000000000032:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.550114+0000", 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.63120186, 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036077209999999999, 2024-09-13T09:17:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1145 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550114+0000", 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550114+0000", 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550112+0000", 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550124+0000", 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.19e-05 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550153+0000", 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8430000000000001e-05 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550191+0000", 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8633000000000001e-05 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550307+0000", 2024-09-13T09:17:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115779 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550552+0000", 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000245262 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550763+0000", 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000211101 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.552233+0000", 2024-09-13T09:17:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014694790000000001 2024-09-13T09:17:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.553567+0000", 2024-09-13T09:17:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013338180000000001 2024-09-13T09:17:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.553632+0000", 2024-09-13T09:17:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4900000000000005e-05 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.553722+0000", 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0476000000000006e-05 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1156 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.562031+0000", 2024-09-13T09:17:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.6192850920000001, 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0034290560000000002, 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1156 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562031+0000", 2024-09-13T09:17:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562031+0000", 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562029+0000", 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562047+0000", 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8093000000000001e-05 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562080+0000", 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3806000000000001e-05 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562099+0000", 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8502999999999999e-05 2024-09-13T09:17:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562185+0000", 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5805999999999998e-05 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562466+0000", 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028099100000000002 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562971+0000", 2024-09-13T09:17:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050485000000000005 2024-09-13T09:17:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.564647+0000", 2024-09-13T09:17:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001675887 2024-09-13T09:17:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.565278+0000", 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00063119800000000002 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.565373+0000", 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5636000000000004e-05 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.565460+0000", 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6798999999999993e-05 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1199 3.6 3:6098d441:::rbd_data.1185192b47c4.0000000000000036:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1158990~65077] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.658149+0000", 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5231674370000001, 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025620700000000001, 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1199 2024-09-13T09:17:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.658149+0000", 2024-09-13T09:17:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.658149+0000", 2024-09-13T09:17:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.658146+0000", 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.658195+0000", 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9063000000000001e-05 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.658233+0000", 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8203e-05 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.658250+0000", 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6736999999999999e-05 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.658365+0000", 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115389 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.658515+0000", 2024-09-13T09:17:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014959199999999999 2024-09-13T09:17:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.658777+0000", 2024-09-13T09:17:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026259100000000001 2024-09-13T09:17:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.660248+0000", 2024-09-13T09:17:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001470809 2024-09-13T09:17:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.660512+0000", 2024-09-13T09:17:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026361800000000002 2024-09-13T09:17:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.660649+0000", 2024-09-13T09:17:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013724200000000001 2024-09-13T09:17:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.660711+0000", 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.198e-05 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1211 3.6 3:6098d441:::rbd_data.1185192b47c4.0000000000000036:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.731677+0000", 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.4496394050000001, 2024-09-13T09:17:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002742516, 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1211 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.731677+0000", 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.731677+0000", 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.731673+0000", 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.731687+0000", 2024-09-13T09:17:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3764999999999999e-05 2024-09-13T09:17:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.731715+0000", 2024-09-13T09:17:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7943999999999999e-05 2024-09-13T09:17:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.731730+0000", 2024-09-13T09:17:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5174000000000001e-05 2024-09-13T09:17:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.731804+0000", 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3749000000000002e-05 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.731929+0000", 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125296 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.732051+0000", 2024-09-13T09:17:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000121916 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.733074+0000", 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010227630000000001 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.734237+0000", 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011634449999999999 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.734315+0000", 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7935999999999994e-05 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.734419+0000", 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010388299999999999 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1255 3.19 3:9bf4724a:::rbd_data.1185192b47c4.0000000000000020:head [set-alloc-hint object_size 4194304 write_size 4194304,write 303047~3719] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.831748+0000", 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3495683299999999, 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027041639999999998, 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1255 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.831748+0000", 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.831748+0000", 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.831746+0000", 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.831760+0000", 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4116e-05 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.831790+0000", 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0046999999999999e-05 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.831807+0000", 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7079999999999999e-05 2024-09-13T09:17:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.831909+0000", 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102211 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.832040+0000", 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013066900000000001 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.832160+0000", 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012034799999999999 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.833446+0000", 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012854279999999999 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834382+0000", 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00093655599999999996 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834422+0000", 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9901000000000003e-05 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834452+0000", 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9947999999999999e-05 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1256 3.19 3:9bf4724a:::rbd_data.1185192b47c4.0000000000000020:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.834936+0000", 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3463803249999999, 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003591136, 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1256 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834936+0000", 2024-09-13T09:17:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834936+0000", 2024-09-13T09:17:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834934+0000", 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834945+0000", 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0890000000000001e-05 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834971+0000", 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5919999999999999e-05 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834983+0000", 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2403e-05 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.835045+0000", 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2323000000000006e-05 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.835105+0000", 2024-09-13T09:17:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9369e-05 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.835517+0000", 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00041229599999999998 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.838427+0000", 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029098589999999999 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.838438+0000", 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1052e-05 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.838493+0000", 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4987000000000001e-05 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.838527+0000", 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3943999999999999e-05 2024-09-13T09:17:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1263 3.19 3:9bf4724a:::rbd_data.1185192b47c4.0000000000000020:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.878974+0000", 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.302342358, 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030362929999999998, 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.277 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1263 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.878974+0000", 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.878974+0000", 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.878971+0000", 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.878986+0000", 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5699e-05 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.879025+0000", 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8617e-05 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.879041+0000", 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6016e-05 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.879186+0000", 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014511600000000001 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.879442+0000", 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025617499999999998 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.879784+0000", 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00034151299999999999 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.881351+0000", 2024-09-13T09:17:35.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015671750000000001 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.881844+0000", 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049293399999999999 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.881919+0000", 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4734999999999995e-05 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.882010+0000", 2024-09-13T09:17:35.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1442999999999998e-05 2024-09-13T09:17:35.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.283 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.283 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1268 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:35.283 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.887012+0000", 2024-09-13T09:17:35.283 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.294303867, 2024-09-13T09:17:35.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031407309999999999, 2024-09-13T09:17:35.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1268 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887012+0000", 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887012+0000", 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887010+0000", 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887025+0000", 2024-09-13T09:17:35.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5667000000000001e-05 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887068+0000", 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2895999999999997e-05 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887091+0000", 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2796e-05 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887204+0000", 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011266599999999999 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887321+0000", 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011748600000000001 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887510+0000", 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018889800000000001 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.889400+0000", 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001889759 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.889939+0000", 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00053940699999999999 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.890056+0000", 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116816 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.890153+0000", 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7013000000000001e-05 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1271 3.4 3:2221ff39:::rbd_data.1185192b47c4.0000000000000004:head [stat,truncate 1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.887118+0000", 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.2941980280000001, 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002935764, 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1271 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887118+0000", 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887118+0000", 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887117+0000", 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887126+0000", 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7940000000000008e-06 2024-09-13T09:17:35.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887148+0000", 2024-09-13T09:17:35.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2402999999999999e-05 2024-09-13T09:17:35.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887164+0000", 2024-09-13T09:17:35.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5736999999999999e-05 2024-09-13T09:17:35.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887252+0000", 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7913999999999999e-05 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887368+0000", 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116239 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887552+0000", 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018406999999999999 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.889459+0000", 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001907249 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.889857+0000", 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039799300000000002 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.889958+0000", 2024-09-13T09:17:35.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010056300000000001 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.890054+0000", 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6151999999999997e-05 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1380 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:head [stat,truncate 1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:34.167107+0000", 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.0142088650000001, 2024-09-13T09:17:35.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030987160000000001, 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1380 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167107+0000", 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167107+0000", 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167104+0000", 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167124+0000", 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9826999999999998e-05 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167170+0000", 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6706000000000003e-05 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167194+0000", 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3156e-05 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167328+0000", 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013484200000000001 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167552+0000", 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022331900000000001 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167924+0000", 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037255300000000002 2024-09-13T09:17:35.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.170038+0000", 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002113967 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.170075+0000", 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6494e-05 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.170153+0000", 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8574999999999999e-05 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.170206+0000", 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2744000000000002e-05 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1383 3.4 3:2221ff39:::rbd_data.1185192b47c4.0000000000000004:head [stat,truncate 1281572] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:34.167225+0000", 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.014090956, 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028776370000000002, 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1383 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:35.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167225+0000", 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167225+0000", 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167224+0000", 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:35.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167233+0000", 2024-09-13T09:17:35.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7439999999999996e-06 2024-09-13T09:17:35.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:35.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167257+0000", 2024-09-13T09:17:35.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.459e-05 2024-09-13T09:17:35.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:35.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167274+0000", 2024-09-13T09:17:35.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6623e-05 2024-09-13T09:17:35.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:35.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167370+0000", 2024-09-13T09:17:35.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6628999999999999e-05 2024-09-13T09:17:35.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:35.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167501+0000", 2024-09-13T09:17:35.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013085899999999999 2024-09-13T09:17:35.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167710+0000", 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020891799999999999 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.169805+0000", 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020950669999999999 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.169910+0000", 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010438900000000001 2024-09-13T09:17:35.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.170034+0000", 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012465600000000001 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.170103+0000", 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8476e-05 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:35.308 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:35.308 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:35.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:17:35.310 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:35.310 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:35.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:35.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:35.312 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:35.302+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:35.315 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:35.316 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:35.336 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:35.336 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:35.357 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:35.357 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:35.378 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:35.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:35.440 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:35.440 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:35.440 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:35.436+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:35.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:17:35.511 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:35.506+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:35.612 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:35.610+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:35.622 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:35.622 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:35.636 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:17:35.713 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:35.706+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:17:35.761 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:35.761 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:35.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:17:35.822 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:35.820+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:35.865 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone1 from 0xeb19641 (overlap 0x3de5e1) to 0x0 2024-09-13T09:17:35.915 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:35.912+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:35.950 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:35.950 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:35.964 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-13T09:17:35.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:17:36.016 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:36.012+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:36.115 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:36.112+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:36.154 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:36.154 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:36.170 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:17:36.216 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:36.214+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:36.220 INFO:teuthology.orchestra.run.smithi032.stdout:598 write 0x2469bca thru 0x2471cf0 (0x8127 bytes) 2024-09-13T09:17:36.236 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:36.236 INFO:teuthology.orchestra.run.smithi032.stdout:601 write 0x8f371af thru 0x8f3771a (0x56c bytes) 2024-09-13T09:17:36.242 INFO:teuthology.orchestra.run.smithi032.stdout:602 read 0x20f4342 thru 0x20ff24f (0xaf0e bytes) 2024-09-13T09:17:36.242 INFO:teuthology.orchestra.run.smithi032.stdout:603 read 0x90a9e40 thru 0x90b0c02 (0x6dc3 bytes) 2024-09-13T09:17:36.242 INFO:teuthology.orchestra.run.smithi032.stdout:604 write 0x6d886e2 thru 0x6d8b45c (0x2d7b bytes) 2024-09-13T09:17:36.250 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:36.251 INFO:teuthology.orchestra.run.smithi032.stdout:606 punch from 0x11f3cef to 0x11fb891, (0x7ba2 bytes) 2024-09-13T09:17:36.251 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:36.251 INFO:teuthology.orchestra.run.smithi032.stdout:609 read 0x4a99a65 thru 0x4a9dfd3 (0x456f bytes) 2024-09-13T09:17:36.251 INFO:teuthology.orchestra.run.smithi032.stdout:613 write 0x16a37b3 thru 0x16acd31 (0x957f bytes) 2024-09-13T09:17:36.258 INFO:teuthology.orchestra.run.smithi032.stdout:614 write 0x7fd42d4 thru 0x7fd67b2 (0x24df bytes) 2024-09-13T09:17:36.265 INFO:teuthology.orchestra.run.smithi032.stdout:615 trunc from 0xeb19641 to 0xe1d89d3 2024-09-13T09:17:36.309 INFO:teuthology.orchestra.run.smithi032.stdout:616 write 0x9449b9c thru 0x944d94a (0x3daf bytes) 2024-09-13T09:17:36.317 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:36.312+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:36.318 INFO:teuthology.orchestra.run.smithi032.stdout:618 trunc from 0xe1d89d3 to 0xcb16c79 2024-09-13T09:17:36.336 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:36.337 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:36.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:17:36.372 INFO:teuthology.orchestra.run.smithi032.stdout:619 writesame 0x409fb27 thru 0x40a5a67 data_size 0x1(0x5f41 bytes) 2024-09-13T09:17:36.400 INFO:teuthology.orchestra.run.smithi032.stdout:620 writesame 0x50d575a thru 0x50de89b data_size 0x1(0x9142 bytes) 2024-09-13T09:17:36.418 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:36.416+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:36.427 INFO:teuthology.orchestra.run.smithi032.stdout:622 punch from 0x8d4f453 to 0x8d5f066, (0xfc13 bytes) 2024-09-13T09:17:36.427 INFO:teuthology.orchestra.run.smithi032.stdout:623 punch from 0x41a8002 to 0x41b08e7, (0x88e5 bytes) 2024-09-13T09:17:36.427 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:36.428 INFO:teuthology.orchestra.run.smithi032.stdout:627 write 0x4946090 thru 0x494cbd6 (0x6b47 bytes) 2024-09-13T09:17:36.433 INFO:teuthology.orchestra.run.smithi032.stdout:629 punch from 0x6b81286 to 0x6b86438, (0x51b2 bytes) 2024-09-13T09:17:36.433 INFO:teuthology.orchestra.run.smithi032.stdout:630 read 0x370d4f thru 0x37543a (0x46ec bytes) 2024-09-13T09:17:36.434 INFO:teuthology.orchestra.run.smithi032.stdout:631 write 0x864fe02 thru 0x86542d0 (0x44cf bytes) 2024-09-13T09:17:36.440 INFO:teuthology.orchestra.run.smithi032.stdout:632 write 0x8bac14b thru 0x8bad975 (0x182b bytes) 2024-09-13T09:17:36.443 INFO:teuthology.orchestra.run.smithi032.stdout:633 read 0x86b1cf5 thru 0x86b44bf (0x27cb bytes) 2024-09-13T09:17:36.443 INFO:teuthology.orchestra.run.smithi032.stdout:634 write 0x33a8162 thru 0x33b5e54 (0xdcf3 bytes) 2024-09-13T09:17:36.448 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:36.449 INFO:teuthology.orchestra.run.smithi032.stdout:636 punch from 0xa1e6dee to 0xa1eb951, (0x4b63 bytes) 2024-09-13T09:17:36.449 INFO:teuthology.orchestra.run.smithi032.stdout:637 write 0xabc2b6d thru 0xabc6004 (0x3498 bytes) 2024-09-13T09:17:36.475 INFO:teuthology.orchestra.run.smithi032.stdout:638 trunc from 0xcb16c79 to 0x48e7a4e 2024-09-13T09:17:36.504 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:36.504 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:36.516 INFO:teuthology.orchestra.run.smithi032.stdout:639 punch from 0x2742baa to 0x274f756, (0xcbac bytes) 2024-09-13T09:17:36.516 INFO:teuthology.orchestra.run.smithi032.stdout:640 read 0x74314c thru 0x74ce7f (0x9d34 bytes) 2024-09-13T09:17:36.519 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:36.516+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:36.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:17:36.545 INFO:teuthology.orchestra.run.smithi032.stdout:641 write 0xa59463f thru 0xa59723e (0x2c00 bytes) 2024-09-13T09:17:36.553 INFO:teuthology.orchestra.run.smithi032.stdout:644 trunc from 0xa59723f to 0x93740ef 2024-09-13T09:17:36.572 INFO:teuthology.orchestra.run.smithi032.stdout:645 punch from 0x7c8aa13 to 0x7c90425, (0x5a12 bytes) 2024-09-13T09:17:36.598 INFO:teuthology.orchestra.run.smithi032.stdout:646 writesame 0xe63b101 thru 0xe63d6ae data_size 0x35(0x25ae bytes) 2024-09-13T09:17:36.598 INFO:teuthology.orchestra.run.smithi032.stdout:647 read 0x203dc93 thru 0x2047290 (0x95fe bytes) 2024-09-13T09:17:36.598 INFO:teuthology.orchestra.run.smithi032.stdout:648 write 0x2a7346c thru 0x2a7fe15 (0xc9aa bytes) 2024-09-13T09:17:36.601 INFO:teuthology.orchestra.run.smithi032.stdout:649 punch from 0x2b68d74 to 0x2b6a7cc, (0x1a58 bytes) 2024-09-13T09:17:36.601 INFO:teuthology.orchestra.run.smithi032.stdout:650 punch from 0x6cea4f1 to 0x6cef984, (0x5493 bytes) 2024-09-13T09:17:36.601 INFO:teuthology.orchestra.run.smithi032.stdout:651 read 0x43aa29c thru 0x43b0ff8 (0x6d5d bytes) 2024-09-13T09:17:36.601 INFO:teuthology.orchestra.run.smithi032.stdout:652 read 0x3aeb5d8 thru 0x3af6ee1 (0xb90a bytes) 2024-09-13T09:17:36.602 INFO:teuthology.orchestra.run.smithi032.stdout:653 read 0x84a987c thru 0x84adb96 (0x431b bytes) 2024-09-13T09:17:36.602 INFO:teuthology.orchestra.run.smithi032.stdout:655 read 0x342ed85 thru 0x3432ed3 (0x414f bytes) 2024-09-13T09:17:36.602 INFO:teuthology.orchestra.run.smithi032.stdout:657 write 0x8709696 thru 0x8717efa (0xe865 bytes) 2024-09-13T09:17:36.609 INFO:teuthology.orchestra.run.smithi032.stdout:658 write 0x326a347 thru 0x3279ddb (0xfa95 bytes) 2024-09-13T09:17:36.615 INFO:teuthology.orchestra.run.smithi032.stdout:659 read 0xb1b3c85 thru 0xb1ba909 (0x6c85 bytes) 2024-09-13T09:17:36.615 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:36.615 INFO:teuthology.orchestra.run.smithi032.stdout:666 write 0x895840e thru 0x89601dc (0x7dcf bytes) 2024-09-13T09:17:36.620 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:36.614+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:36.620 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:36.620 INFO:teuthology.orchestra.run.smithi032.stdout:670 writesame 0x58fa88d thru 0x5908ac9 data_size 0x1(0xe23d bytes) 2024-09-13T09:17:36.654 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:36.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:17:36.671 INFO:teuthology.orchestra.run.smithi032.stdout:672 trunc from 0xe63d6af to 0xb16053a 2024-09-13T09:17:36.683 INFO:teuthology.orchestra.run.smithi032.stdout:676 write 0x2a29a4c thru 0x2a36eb9 (0xd46e bytes) 2024-09-13T09:17:36.686 INFO:teuthology.orchestra.run.smithi032.stdout:677 punch from 0x91ab04f to 0x91ac059, (0x100a bytes) 2024-09-13T09:17:36.686 INFO:teuthology.orchestra.run.smithi032.stdout:678 write 0x3178eff thru 0x3179dd5 (0xed7 bytes) 2024-09-13T09:17:36.692 INFO:teuthology.orchestra.run.smithi032.stdout:679 read 0x42f8b15 thru 0x430662d (0xdb19 bytes) 2024-09-13T09:17:36.693 INFO:teuthology.orchestra.run.smithi032.stdout:680 punch from 0x59fcd67 to 0x5a03fec, (0x7285 bytes) 2024-09-13T09:17:36.693 INFO:teuthology.orchestra.run.smithi032.stdout:683 writesame 0xad08188 thru 0xad0e9f6 data_size 0x1(0x686f bytes) 2024-09-13T09:17:36.720 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:36.716+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:36.747 INFO:teuthology.orchestra.run.smithi032.stdout:684 trunc from 0xb16053a to 0x5a57422 2024-09-13T09:17:36.791 INFO:teuthology.orchestra.run.smithi032.stdout:685 trunc from 0x5a57422 to 0xe9d3d5e 2024-09-13T09:17:36.815 INFO:teuthology.orchestra.run.smithi032.stdout:686 trunc from 0xe9d3d5e to 0xa123b49 2024-09-13T09:17:36.821 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:36.814+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:36.828 INFO:teuthology.orchestra.run.smithi032.stdout:687 write 0x1ce28fb thru 0x1cf0990 (0xe096 bytes) 2024-09-13T09:17:36.836 INFO:teuthology.orchestra.run.smithi032.stdout:688 writesame 0x9c9516f thru 0x9ca2cdf data_size 0x1(0xdb71 bytes) 2024-09-13T09:17:36.851 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:36.851 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:36.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:17:36.905 INFO:teuthology.orchestra.run.smithi032.stdout:691 read 0x8a9c532 thru 0x8aabb85 (0xf654 bytes) 2024-09-13T09:17:36.906 INFO:teuthology.orchestra.run.smithi032.stdout:692 trunc from 0xa123b49 to 0x97cc665 2024-09-13T09:17:36.921 INFO:teuthology.orchestra.run.smithi032.stdout:693 punch from 0x1ca23b5 to 0x1ca3c65, (0x18b0 bytes) 2024-09-13T09:17:36.922 INFO:teuthology.orchestra.run.smithi032.stdout:695 punch from 0x2409571 to 0x240fc85, (0x6714 bytes) 2024-09-13T09:17:36.922 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:36.922 INFO:teuthology.orchestra.run.smithi032.stdout:698 read 0x1522d11 thru 0x152b0d7 (0x83c7 bytes) 2024-09-13T09:17:36.922 INFO:teuthology.orchestra.run.smithi032.stdout:700 punch from 0x406cfc to 0x40cb02, (0x5e06 bytes) 2024-09-13T09:17:36.923 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:36.923 INFO:teuthology.orchestra.run.smithi032.stdout:707 writesame 0x96297e thru 0x971048 data_size 0x1(0xe6cb bytes) 2024-09-13T09:17:36.953 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:36.950+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:36.995 INFO:teuthology.orchestra.run.smithi032.stdout:716 write 0x3b46e0f thru 0x3b52cd2 (0xbec4 bytes) 2024-09-13T09:17:37.021 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:37.018+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:37.031 INFO:teuthology.orchestra.run.smithi032.stdout:718 writesame 0xed38c96 thru 0xed456fe data_size 0x1(0xca69 bytes) 2024-09-13T09:17:37.031 INFO:teuthology.orchestra.run.smithi032.stdout:719 read 0x84c315e thru 0x84ce41f (0xb2c2 bytes) 2024-09-13T09:17:37.032 INFO:teuthology.orchestra.run.smithi032.stdout:720 write 0x88ecec5 thru 0x88fb465 (0xe5a1 bytes) 2024-09-13T09:17:37.039 INFO:teuthology.orchestra.run.smithi032.stdout:721 write 0x5a3006b thru 0x5a3a0b4 (0xa04a bytes) 2024-09-13T09:17:37.040 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:37.040 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:37.042 INFO:teuthology.orchestra.run.smithi032.stdout:722 read 0x21aa42 thru 0x223885 (0x8e44 bytes) 2024-09-13T09:17:37.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:17:37.078 INFO:teuthology.orchestra.run.smithi032.stdout:725 trunc from 0xed456ff to 0x13b7355 2024-09-13T09:17:37.118 INFO:teuthology.orchestra.run.smithi032.stdout:726 read 0x28498b thru 0x289ef5 (0x556b bytes) 2024-09-13T09:17:37.119 INFO:teuthology.orchestra.run.smithi032.stdout:727 punch from 0xd90564 to 0xd98505, (0x7fa1 bytes) 2024-09-13T09:17:37.119 INFO:teuthology.orchestra.run.smithi032.stdout:729 read 0xef9ce8 thru 0xf06039 (0xc352 bytes) 2024-09-13T09:17:37.120 INFO:teuthology.orchestra.run.smithi032.stdout:730 punch from 0x4921fe to 0x497d70, (0x5b72 bytes) 2024-09-13T09:17:37.120 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:37.141 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:37.138+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:37.165 INFO:teuthology.orchestra.run.smithi032.stdout:732 write 0xeb9db50 thru 0xebad891 (0xfd42 bytes) 2024-09-13T09:17:37.171 INFO:teuthology.orchestra.run.smithi032.stdout:733 read 0x3961087 thru 0x39628b5 (0x182f bytes) 2024-09-13T09:17:37.172 INFO:teuthology.orchestra.run.smithi032.stdout:734 write 0x3ed3862 thru 0x3ee277a (0xef19 bytes) 2024-09-13T09:17:37.178 INFO:teuthology.orchestra.run.smithi032.stdout:735 read 0x25b3350 thru 0x25b447a (0x112b bytes) 2024-09-13T09:17:37.178 INFO:teuthology.orchestra.run.smithi032.stdout:736 read 0x28296de thru 0x282eb05 (0x5428 bytes) 2024-09-13T09:17:37.178 INFO:teuthology.orchestra.run.smithi032.stdout:737 write 0x4dd54d6 thru 0x4ddc5fa (0x7125 bytes) 2024-09-13T09:17:37.185 INFO:teuthology.orchestra.run.smithi032.stdout:738 write 0x470474 thru 0x4776b3 (0x7240 bytes) 2024-09-13T09:17:37.187 INFO:teuthology.orchestra.run.smithi032.stdout:739 clone 3 order 18 su 4096 sc 14 2024-09-13T09:17:37.221 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:37.221 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:37.223 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:37.220+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:37.235 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:17:37.326 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:37.322+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:37.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:17:37.402 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:37.402 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:37.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:17:37.446 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:37.444+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:37.526 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:37.524+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:37.558 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:37.559 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:37.575 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:37.575 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:37.586 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:37.586 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:37.591 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:37.591 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:37.601 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:17:37.610 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:37.610 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:37.627 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:37.624+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:37.629 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:37.647 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:37.667 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:37.667 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:37.686 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:37.686 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:37.731 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:37.731 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:37.731 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:17:37.731 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:17:37.731 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:37.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:17:37.750 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:37.748+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:37.773 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:17:37.829 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:37.828+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:37.849 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone2 from 0xebad892 (overlap 0x13b7355) to 0x9db9aa 2024-09-13T09:17:37.893 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:37.894 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:17:37.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:17:37.894 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:17:37.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.894 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:1066 3.10 3:0d62146f:::rbd_data.1185192b47c4.0000000000000026:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3145728~851] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:37.894 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:33.365917+0000", 2024-09-13T09:17:37.894 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.5246619130000001, 2024-09-13T09:17:37.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024188679999999998, 2024-09-13T09:17:37.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1066 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.365917+0000", 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.365917+0000", 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.365914+0000", 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.365932+0000", 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8490000000000001e-05 2024-09-13T09:17:37.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.365971+0000", 2024-09-13T09:17:37.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8182999999999997e-05 2024-09-13T09:17:37.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.365994+0000", 2024-09-13T09:17:37.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3847000000000001e-05 2024-09-13T09:17:37.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.366069+0000", 2024-09-13T09:17:37.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4197999999999997e-05 2024-09-13T09:17:37.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.366221+0000", 2024-09-13T09:17:37.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015278199999999999 2024-09-13T09:17:37.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:37.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.366358+0000", 2024-09-13T09:17:37.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013694099999999999 2024-09-13T09:17:37.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.367874+0000", 2024-09-13T09:17:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015155139999999999 2024-09-13T09:17:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.368203+0000", 2024-09-13T09:17:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032940399999999999 2024-09-13T09:17:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.368284+0000", 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0289000000000003e-05 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.368336+0000", 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2540000000000002e-05 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:1143 3.10 3:0d62146f:::rbd_data.1185192b47c4.0000000000000026:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:33.547962+0000", 2024-09-13T09:17:37.901 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.342617422, 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026864329999999998, 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1143 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.547962+0000", 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.547962+0000", 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.547960+0000", 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.547970+0000", 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0753e-05 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.547995+0000", 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4899999999999999e-05 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.548006+0000", 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0273e-05 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.548088+0000", 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2355999999999996e-05 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.548241+0000", 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015330899999999999 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.548420+0000", 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000178915 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.549575+0000", 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011552369999999999 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.550566+0000", 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099056699999999992 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.550613+0000", 2024-09-13T09:17:37.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7293000000000003e-05 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.550648+0000", 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4921e-05 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:1158 3.3 e28/22 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:head v 28'1, mlcod=0'0)", 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:33.563433+0000", 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.3271466710000004, 2024-09-13T09:17:37.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001394143, 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.563433+0000", 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.563433+0000", 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.563430+0000", 2024-09-13T09:17:37.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:37.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.563449+0000", 2024-09-13T09:17:37.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9018999999999999e-05 2024-09-13T09:17:37.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.563490+0000", 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0559999999999998e-05 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.563533+0000", 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2939999999999999e-05 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.563602+0000", 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8901999999999999e-05 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.563621+0000", 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9432999999999998e-05 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.564725+0000", 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011035280000000001 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.564827+0000", 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010204699999999999 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:1207 3.a 3:548359ed:::rbd_data.1185192b47c4.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:33.697882+0000", 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.1926976820000004, 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030789060000000002, 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1207 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.697882+0000", 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.697882+0000", 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.697879+0000", 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.697898+0000", 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9392999999999999e-05 2024-09-13T09:17:37.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.697932+0000", 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4017000000000003e-05 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.697949+0000", 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6616000000000002e-05 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.698043+0000", 2024-09-13T09:17:37.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4673000000000006e-05 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.698282+0000", 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000238371 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.698553+0000", 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027166100000000003 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700302+0000", 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017481860000000001 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700783+0000", 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00048154699999999998 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700864+0000", 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0778999999999993e-05 2024-09-13T09:17:37.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700960+0000", 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6545999999999993e-05 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4485.0:1207 3.a e28/22 ondisk, result = 0)", 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:33.700622+0000", 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.1899570600000002, 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035215700000000002, 2024-09-13T09:17:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:37.919 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700622+0000", 2024-09-13T09:17:37.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700622+0000", 2024-09-13T09:17:37.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700621+0000", 2024-09-13T09:17:37.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:37.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700631+0000", 2024-09-13T09:17:37.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0356e-05 2024-09-13T09:17:37.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700656+0000", 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5063e-05 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700669+0000", 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2449999999999999e-05 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700757+0000", 2024-09-13T09:17:37.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8605999999999998e-05 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700771+0000", 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700974+0000", 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020311099999999999 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:1210 3.a 3:548359ed:::rbd_data.1185192b47c4.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:33.729887+0000", 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.1606920460000003, 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051157470000000003, 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1210 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.923 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.729887+0000", 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.729887+0000", 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.729884+0000", 2024-09-13T09:17:37.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:37.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.729901+0000", 2024-09-13T09:17:37.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7373000000000001e-05 2024-09-13T09:17:37.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.729945+0000", 2024-09-13T09:17:37.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3226000000000003e-05 2024-09-13T09:17:37.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.729968+0000", 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3822999999999999e-05 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.730081+0000", 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113066 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.730093+0000", 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1727e-05 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.732734+0000", 2024-09-13T09:17:37.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026410800000000001 2024-09-13T09:17:37.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.732816+0000", 2024-09-13T09:17:37.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1291999999999993e-05 2024-09-13T09:17:37.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:37.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.732918+0000", 2024-09-13T09:17:37.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102373 2024-09-13T09:17:37.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:37.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.734007+0000", 2024-09-13T09:17:37.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001089338 2024-09-13T09:17:37.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.734873+0000", 2024-09-13T09:17:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00086618600000000002 2024-09-13T09:17:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.734951+0000", 2024-09-13T09:17:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7440000000000004e-05 2024-09-13T09:17:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.735003+0000", 2024-09-13T09:17:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2055999999999998e-05 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:1272 3.3 e29/22 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:head v 29'2, mlcod=0'0)", 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:33.886797+0000", 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0037824899999999, 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001340614, 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.886797+0000", 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.886797+0000", 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.886794+0000", 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:37.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.886814+0000", 2024-09-13T09:17:37.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9896000000000001e-05 2024-09-13T09:17:37.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.886856+0000", 2024-09-13T09:17:37.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.189e-05 2024-09-13T09:17:37.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.886877+0000", 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1325999999999999e-05 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.887076+0000", 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019917900000000001 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.887131+0000", 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4052999999999999e-05 2024-09-13T09:17:37.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.888086+0000", 2024-09-13T09:17:37.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095573499999999998 2024-09-13T09:17:37.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.888137+0000", 2024-09-13T09:17:37.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1044999999999997e-05 2024-09-13T09:17:37.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.936 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.936 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:1384 3.3 e29/22 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:head v 29'3, mlcod=28'1)", 2024-09-13T09:17:37.936 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:34.167127+0000", 2024-09-13T09:17:37.936 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.723451887, 2024-09-13T09:17:37.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016876840000000001, 2024-09-13T09:17:37.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.937 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.167127+0000", 2024-09-13T09:17:37.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.167127+0000", 2024-09-13T09:17:37.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.167125+0000", 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.167147+0000", 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2033000000000001e-05 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.167190+0000", 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2645999999999998e-05 2024-09-13T09:17:37.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.167212+0000", 2024-09-13T09:17:37.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2527e-05 2024-09-13T09:17:37.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.167423+0000", 2024-09-13T09:17:37.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000210788 2024-09-13T09:17:37.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.167489+0000", 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5863000000000003e-05 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.168699+0000", 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012097640000000001 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.168815+0000", 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011638599999999999 2024-09-13T09:17:37.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:1475 3.10 3:0d62146f:::rbd_data.1185192b47c4.0000000000000026:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1590581~6967] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:34.402977+0000", 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.4876026910000002, 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030337139999999999, 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:37.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1475 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.402977+0000", 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.402977+0000", 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.402974+0000", 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.402999+0000", 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5483e-05 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.403035+0000", 2024-09-13T09:17:37.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5777e-05 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.403051+0000", 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6543000000000001e-05 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.403165+0000", 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011364199999999999 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.403303+0000", 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138183 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.403555+0000", 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00025172399999999999 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.405656+0000", 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021008329999999999 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:37.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.405870+0000", 2024-09-13T09:17:37.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021389900000000001 2024-09-13T09:17:37.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.405955+0000", 2024-09-13T09:17:37.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5499000000000003e-05 2024-09-13T09:17:37.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.406010+0000", 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5117000000000001e-05 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4485.0:1475 3.10 e29/22 ondisk, result = 0)", 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:34.405684+0000", 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.4848950520000002, 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033660500000000002, 2024-09-13T09:17:37.947 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.405684+0000", 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.405684+0000", 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.405682+0000", 2024-09-13T09:17:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.405696+0000", 2024-09-13T09:17:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.361e-05 2024-09-13T09:17:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.405726+0000", 2024-09-13T09:17:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0577000000000002e-05 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.405740+0000", 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3896e-05 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.405849+0000", 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010854299999999999 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.405861+0000", 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.222e-05 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.406021+0000", 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015974199999999999 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:1477 3.10 3:0d62146f:::rbd_data.1185192b47c4.0000000000000026:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:34.421967+0000", 2024-09-13T09:17:37.951 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.4686121480000001, 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024634460000000002, 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1477 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.421967+0000", 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.421967+0000", 2024-09-13T09:17:37.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.421965+0000", 2024-09-13T09:17:37.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.421977+0000", 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1946e-05 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.422009+0000", 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1770000000000002e-05 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.422026+0000", 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7337e-05 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.422154+0000", 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012816600000000001 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.422283+0000", 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012884499999999999 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.422434+0000", 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015101599999999999 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.423525+0000", 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010912750000000001 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.424306+0000", 2024-09-13T09:17:37.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00078076399999999998 2024-09-13T09:17:37.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.424378+0000", 2024-09-13T09:17:37.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1632999999999997e-05 2024-09-13T09:17:37.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.424431+0000", 2024-09-13T09:17:37.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2800000000000003e-05 2024-09-13T09:17:37.957 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.957 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4485.0:1477 3.10 e29/22 ondisk, result = 0)", 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:34.424133+0000", 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.4664464069999998, 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030809400000000002, 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.424133+0000", 2024-09-13T09:17:37.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.424133+0000", 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.424131+0000", 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.424142+0000", 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0633e-05 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.424170+0000", 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8566999999999999e-05 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.424184+0000", 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4059e-05 2024-09-13T09:17:37.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.424282+0000", 2024-09-13T09:17:37.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7462999999999998e-05 2024-09-13T09:17:37.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.424297+0000", 2024-09-13T09:17:37.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5512999999999999e-05 2024-09-13T09:17:37.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.424441+0000", 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014356199999999999 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:1522 3.a 3:548359ed:::rbd_data.1185192b47c4.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1994137~18216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:34.524257+0000", 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.3663218329999998, 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031142420000000001, 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1522 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524257+0000", 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524257+0000", 2024-09-13T09:17:37.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524255+0000", 2024-09-13T09:17:37.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:37.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524305+0000", 2024-09-13T09:17:37.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9509999999999999e-05 2024-09-13T09:17:37.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524338+0000", 2024-09-13T09:17:37.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3596999999999997e-05 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524353+0000", 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5296000000000001e-05 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524488+0000", 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013414199999999999 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524725+0000", 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023750900000000001 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.525136+0000", 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041086300000000003 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.526705+0000", 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015691310000000001 2024-09-13T09:17:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527242+0000", 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00053680199999999996 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527318+0000", 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6156999999999996e-05 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527372+0000", 2024-09-13T09:17:37.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3562e-05 2024-09-13T09:17:37.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.969 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.969 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4485.0:1522 3.a e29/22 ondisk, result = 0)", 2024-09-13T09:17:37.969 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:34.527086+0000", 2024-09-13T09:17:37.969 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.3634928849999999, 2024-09-13T09:17:37.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029843799999999999, 2024-09-13T09:17:37.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.969 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527086+0000", 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527086+0000", 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527084+0000", 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527096+0000", 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.119e-05 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527122+0000", 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6927000000000002e-05 2024-09-13T09:17:37.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527135+0000", 2024-09-13T09:17:37.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2807e-05 2024-09-13T09:17:37.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527218+0000", 2024-09-13T09:17:37.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2775000000000006e-05 2024-09-13T09:17:37.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527234+0000", 2024-09-13T09:17:37.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5523999999999999e-05 2024-09-13T09:17:37.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527385+0000", 2024-09-13T09:17:37.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015121899999999999 2024-09-13T09:17:37.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.973 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:1525 3.a 3:548359ed:::rbd_data.1185192b47c4.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:34.544428+0000", 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.346151103, 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0033108690000000001, 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1525 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544428+0000", 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544428+0000", 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544426+0000", 2024-09-13T09:17:37.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544438+0000", 2024-09-13T09:17:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2717e-05 2024-09-13T09:17:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544471+0000", 2024-09-13T09:17:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2592999999999998e-05 2024-09-13T09:17:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544505+0000", 2024-09-13T09:17:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4409999999999998e-05 2024-09-13T09:17:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544630+0000", 2024-09-13T09:17:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124769 2024-09-13T09:17:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544853+0000", 2024-09-13T09:17:37.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000223255 2024-09-13T09:17:37.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:37.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.545134+0000", 2024-09-13T09:17:37.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028065499999999999 2024-09-13T09:17:37.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:37.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.546554+0000", 2024-09-13T09:17:37.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014199390000000001 2024-09-13T09:17:37.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:37.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547592+0000", 2024-09-13T09:17:37.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010378049999999999 2024-09-13T09:17:37.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547672+0000", 2024-09-13T09:17:37.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9806000000000001e-05 2024-09-13T09:17:37.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547739+0000", 2024-09-13T09:17:37.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7290000000000001e-05 2024-09-13T09:17:37.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.980 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.980 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4485.0:1525 3.a e29/22 ondisk, result = 0)", 2024-09-13T09:17:37.980 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:34.547357+0000", 2024-09-13T09:17:37.980 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.3432226809999999, 2024-09-13T09:17:37.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00039517699999999997, 2024-09-13T09:17:37.981 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:37.981 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547357+0000", 2024-09-13T09:17:37.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547357+0000", 2024-09-13T09:17:37.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547355+0000", 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547369+0000", 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4477e-05 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547407+0000", 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7345999999999998e-05 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547428+0000", 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1022999999999998e-05 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547553+0000", 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012529199999999999 2024-09-13T09:17:37.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547583+0000", 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9624e-05 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547752+0000", 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000169109 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.985 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:1589 3.3 e29/22 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:head v 29'4, mlcod=29'2)", 2024-09-13T09:17:37.985 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:34.754397+0000", 2024-09-13T09:17:37.985 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.1361822359999998, 2024-09-13T09:17:37.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001484485, 2024-09-13T09:17:37.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.985 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.754397+0000", 2024-09-13T09:17:37.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.754397+0000", 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.754394+0000", 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.754414+0000", 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9966000000000002e-05 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.754463+0000", 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9316e-05 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.754486+0000", 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2836999999999998e-05 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.754620+0000", 2024-09-13T09:17:37.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013373899999999999 2024-09-13T09:17:37.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.754676+0000", 2024-09-13T09:17:37.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6226e-05 2024-09-13T09:17:37.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.755799+0000", 2024-09-13T09:17:37.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011222750000000001 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.755881+0000", 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2916000000000004e-05 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:2204 3.1e e31/22 3:7c0417ed:::rbd_object_map.1185be6cd1eb.0000000000000006:head v 31'17, mlcod=30'15)", 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:37.758072+0000", 2024-09-13T09:17:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.132507545, 2024-09-13T09:17:37.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00086078000000000001, 2024-09-13T09:17:37.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.990 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.758072+0000", 2024-09-13T09:17:37.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.758072+0000", 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.758071+0000", 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.758077+0000", 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5639999999999996e-06 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.758092+0000", 2024-09-13T09:17:37.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5271e-05 2024-09-13T09:17:37.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.758102+0000", 2024-09-13T09:17:37.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6700000000000006e-06 2024-09-13T09:17:37.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.758165+0000", 2024-09-13T09:17:37.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2824999999999996e-05 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.758181+0000", 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6161e-05 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.758826+0000", 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00064518400000000004 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.758932+0000", 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106406 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:2266 3.1e e31/22 3:78e6a0e6:::rbd_object_map.118591b29fad:head v 31'18, mlcod=30'16)", 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:37.819861+0000", 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.070718239000000002, 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011288260000000001, 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:37.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.819861+0000", 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.819861+0000", 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.819859+0000", 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.819871+0000", 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1999e-05 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.819890+0000", 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9151999999999999e-05 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:37.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.819900+0000", 2024-09-13T09:17:37.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1810000000000007e-06 2024-09-13T09:17:37.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:37.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.819960+0000", 2024-09-13T09:17:37.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0299e-05 2024-09-13T09:17:37.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:37.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.819988+0000", 2024-09-13T09:17:37.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7781999999999999e-05 2024-09-13T09:17:37.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:37.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.820945+0000", 2024-09-13T09:17:37.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095721300000000005 2024-09-13T09:17:37.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:37.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.820990+0000", 2024-09-13T09:17:37.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4968e-05 2024-09-13T09:17:37.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.998 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:37.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:2281 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:37.834226+0000", 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.056353409, 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015212509999999999, 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2281 2024-09-13T09:17:37.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.834226+0000", 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.834226+0000", 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.834223+0000", 2024-09-13T09:17:38.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:38.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.834241+0000", 2024-09-13T09:17:38.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.855e-05 2024-09-13T09:17:38.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.834277+0000", 2024-09-13T09:17:38.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5345999999999997e-05 2024-09-13T09:17:38.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.834292+0000", 2024-09-13T09:17:38.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.571e-05 2024-09-13T09:17:38.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.834363+0000", 2024-09-13T09:17:38.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1087999999999994e-05 2024-09-13T09:17:38.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.834435+0000", 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1767000000000006e-05 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.834593+0000", 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000157343 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.835239+0000", 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00064652199999999996 2024-09-13T09:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.835677+0000", 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000438057 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.835722+0000", 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5222000000000002e-05 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.835747+0000", 2024-09-13T09:17:38.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4649000000000001e-05 2024-09-13T09:17:38.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.005 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.005 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.005 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:38.007 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:17:38.008 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:38.002+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:38.008 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-13T09:17:38.008 INFO:teuthology.orchestra.run.smithi032.stdout:740 punch from 0xb8329a4 to 0xb83f015, (0xc671 bytes) 2024-09-13T09:17:38.008 INFO:teuthology.orchestra.run.smithi032.stdout:741 write 0xa200d82 thru 0xa208276 (0x74f5 bytes) 2024-09-13T09:17:38.008 INFO:teuthology.orchestra.run.smithi032.stdout:743 read 0x60dcd5e thru 0x60e75f2 (0xa895 bytes) 2024-09-13T09:17:38.008 INFO:teuthology.orchestra.run.smithi032.stdout:744 write 0x8028c53 thru 0x80363cd (0xd77b bytes) 2024-09-13T09:17:38.008 INFO:teuthology.orchestra.run.smithi032.stdout:745 punch from 0xb278a1a to 0xb27d4b6, (0x4a9c bytes) 2024-09-13T09:17:38.008 INFO:teuthology.orchestra.run.smithi032.stdout:746 writesame 0x4826108 thru 0x482ec5b data_size 0x4a(0x8b54 bytes) 2024-09-13T09:17:38.015 INFO:teuthology.orchestra.run.smithi032.stdout:747 read 0x419e4f8 thru 0x419e59c (0xa5 bytes) 2024-09-13T09:17:38.015 INFO:teuthology.orchestra.run.smithi032.stdout:748 write 0xd31e3dc thru 0xd327897 (0x94bc bytes) 2024-09-13T09:17:38.030 INFO:teuthology.orchestra.run.smithi032.stdout:749 write 0x4bd8199 thru 0x4bddd0c (0x5b74 bytes) 2024-09-13T09:17:38.039 INFO:teuthology.orchestra.run.smithi032.stdout:750 write 0x943e3b4 thru 0x94489c5 (0xa612 bytes) 2024-09-13T09:17:38.053 INFO:teuthology.orchestra.run.smithi032.stdout:751 write 0xe8660a2 thru 0xe8751f6 (0xf155 bytes) 2024-09-13T09:17:38.070 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:38.070 INFO:teuthology.orchestra.run.smithi032.stdout:753 read 0xe55a8c0 thru 0xe5668f2 (0xc033 bytes) 2024-09-13T09:17:38.070 INFO:teuthology.orchestra.run.smithi032.stdout:754 write 0x14b8f43 thru 0x14b8f8c (0x4a bytes) 2024-09-13T09:17:38.078 INFO:teuthology.orchestra.run.smithi032.stdout:755 write 0x4824739 thru 0x4824c6b (0x533 bytes) 2024-09-13T09:17:38.082 INFO:teuthology.orchestra.run.smithi032.stdout:756 punch from 0x70eeddf to 0x70ef0d3, (0x2f4 bytes) 2024-09-13T09:17:38.096 INFO:teuthology.orchestra.run.smithi032.stdout:757 trunc from 0xebad892 to 0x49e2f6a 2024-09-13T09:17:38.108 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:38.102+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:38.136 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:38.136 INFO:teuthology.orchestra.run.smithi032.stdout:759 read 0x406d5af thru 0x407b572 (0xdfc4 bytes) 2024-09-13T09:17:38.156 INFO:teuthology.orchestra.run.smithi032.stdout:760 write 0xcb07155 thru 0xcb11c16 (0xaac2 bytes) 2024-09-13T09:17:38.170 INFO:teuthology.orchestra.run.smithi032.stdout:761 read 0x1d2a459 thru 0x1d3603a (0xbbe2 bytes) 2024-09-13T09:17:38.170 INFO:teuthology.orchestra.run.smithi032.stdout:763 punch from 0x35f6eac to 0x3604d73, (0xdec7 bytes) 2024-09-13T09:17:38.170 INFO:teuthology.orchestra.run.smithi032.stdout:764 punch from 0x8e690ba to 0x8e7093e, (0x7884 bytes) 2024-09-13T09:17:38.170 INFO:teuthology.orchestra.run.smithi032.stdout:765 read 0xa297a4d thru 0xa29be12 (0x43c6 bytes) 2024-09-13T09:17:38.185 INFO:teuthology.orchestra.run.smithi032.stdout:766 trunc from 0xcb11c17 to 0x28d15f7 2024-09-13T09:17:38.193 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:38.194 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:17:38.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.194 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:2774 3.11 3:8acd209c:::rbd_data.118591b29fad.0000000000000128:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:38.194 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.190851+0000", 2024-09-13T09:17:38.194 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0023233989999999999, 2024-09-13T09:17:38.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0023681090000000002, 2024-09-13T09:17:38.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:17:38.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:38.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:38.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2774 2024-09-13T09:17:38.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.195 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.190851+0000", 2024-09-13T09:17:38.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.190851+0000", 2024-09-13T09:17:38.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.190850+0000", 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.190859+0000", 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5219999999999996e-06 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.190872+0000", 2024-09-13T09:17:38.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2959000000000001e-05 2024-09-13T09:17:38.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.190882+0000", 2024-09-13T09:17:38.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8600000000000005e-06 2024-09-13T09:17:38.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.190998+0000", 2024-09-13T09:17:38.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011592200000000001 2024-09-13T09:17:38.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.191091+0000", 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3163000000000002e-05 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.191465+0000", 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037375899999999998 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193189+0000", 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001724281 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4485.0:2774 3.11 e31/22 ondisk, result = 0)", 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.193320+0000", 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9998546, 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013003099999999999, 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:38.199 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193320+0000", 2024-09-13T09:17:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193320+0000", 2024-09-13T09:17:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193319+0000", 2024-09-13T09:17:38.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:38.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193324+0000", 2024-09-13T09:17:38.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5430000000000001e-06 2024-09-13T09:17:38.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193335+0000", 2024-09-13T09:17:38.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0998999999999999e-05 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193342+0000", 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7360000000000004e-06 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193397+0000", 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4870000000000002e-05 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193407+0000", 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0127e-05 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:2782 3.13 3:ccb08fbc:::rbd_data.118591b29fad.0000000000000130:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.193361+0000", 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9998131, 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000233188, 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:17:38.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2782 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193361+0000", 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193361+0000", 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193360+0000", 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193365+0000", 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9840000000000001e-06 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193375+0000", 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6430000000000007e-06 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193382+0000", 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3459999999999996e-06 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.193485+0000", 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102953 2024-09-13T09:17:38.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.207 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.207 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-13T09:17:38.207 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 3 2024-09-13T09:17:38.207 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:38.208 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:38.204+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:38.215 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:17:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:767 read 0x12b854c thru 0x12bc544 (0x3ff9 bytes) 2024-09-13T09:17:38.241 INFO:teuthology.orchestra.run.smithi032.stdout:768 write 0x48befe5 thru 0x48cbb6e (0xcb8a bytes) 2024-09-13T09:17:38.266 INFO:teuthology.orchestra.run.smithi032.stdout:769 read 0x3a7755e thru 0x3a8671b (0xf1be bytes) 2024-09-13T09:17:38.266 INFO:teuthology.orchestra.run.smithi032.stdout:770 punch from 0xc29e64 to 0xc3182d, (0x79c9 bytes) 2024-09-13T09:17:38.266 INFO:teuthology.orchestra.run.smithi032.stdout:771 read 0x254608c thru 0x254813b (0x20b0 bytes) 2024-09-13T09:17:38.272 INFO:teuthology.orchestra.run.smithi032.stdout:772 trunc from 0x48cbb6f to 0x7bb3788 2024-09-13T09:17:38.293 INFO:teuthology.orchestra.run.smithi032.stdout:774 trunc from 0x7bb3788 to 0x24818ed 2024-09-13T09:17:38.309 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:38.304+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:38.352 INFO:teuthology.orchestra.run.smithi032.stdout:775 punch from 0x118c173 to 0x118d072, (0xeff bytes) 2024-09-13T09:17:38.352 INFO:teuthology.orchestra.run.smithi032.stdout:777 punch from 0x15a08f6 to 0x15a14f4, (0xbfe bytes) 2024-09-13T09:17:38.374 INFO:teuthology.orchestra.run.smithi032.stdout:779 write 0x82a501d thru 0x82b43cc (0xf3b0 bytes) 2024-09-13T09:17:38.403 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:38.403 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:38.403 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:17:38.403 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:17:38.404 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:38.406 INFO:teuthology.orchestra.run.smithi032.stdout:780 read 0x259400e thru 0x25a0a2e (0xca21 bytes) 2024-09-13T09:17:38.410 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:38.408+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:38.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:17:38.430 INFO:teuthology.orchestra.run.smithi032.stdout:781 write 0xd5ccae3 thru 0xd5cefcd (0x24eb bytes) 2024-09-13T09:17:38.462 INFO:teuthology.orchestra.run.smithi032.stdout:783 trunc from 0xd5cefce to 0x2af7152 2024-09-13T09:17:38.511 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:38.506+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:38.512 INFO:teuthology.orchestra.run.smithi032.stdout:784 writesame 0x8d74bd8 thru 0x8d7d97d data_size 0x1(0x8da6 bytes) 2024-09-13T09:17:38.512 INFO:teuthology.orchestra.run.smithi032.stdout:786 read 0x7a27b98 thru 0x7a32e5e (0xb2c7 bytes) 2024-09-13T09:17:38.516 INFO:teuthology.orchestra.run.smithi032.stdout:787 trunc from 0x8d7d97e to 0xa9e736b 2024-09-13T09:17:38.529 INFO:teuthology.orchestra.run.smithi032.stdout:788 read 0x9c184e thru 0x9cd50c (0xbcbf bytes) 2024-09-13T09:17:38.540 INFO:teuthology.orchestra.run.smithi032.stdout:789 trunc from 0xa9e736b to 0x7edcc70 2024-09-13T09:17:38.552 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:38.552 INFO:teuthology.orchestra.run.smithi032.stdout:791 write 0x213686a thru 0x213beca (0x5661 bytes) 2024-09-13T09:17:38.558 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:38.558 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:17:38.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:17:38.558 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:17:38.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.162655815000001, 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077478195999999999, 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392636+0000", 2024-09-13T09:17:38.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392651+0000", 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392674+0000", 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.264e-05 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392691+0000", 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7203000000000001e-05 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392780+0000", 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8671999999999994e-05 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392794+0000", 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.466664+0000", 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073869913999999995 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467139+0000", 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047488599999999999 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467664+0000", 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052530600000000002 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469125+0000", 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001460432 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469847+0000", 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072197799999999996 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470006+0000", 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015955900000000001 2024-09-13T09:17:38.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470116+0000", 2024-09-13T09:17:38.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109836 2024-09-13T09:17:38.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.565 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.565 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.565 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531091+0000", 2024-09-13T09:17:38.565 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.024202843000001, 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010986811000000001, 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 34 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531091+0000", 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531091+0000", 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531090+0000", 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531096+0000", 2024-09-13T09:17:38.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.806e-06 2024-09-13T09:17:38.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531108+0000", 2024-09-13T09:17:38.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.224e-05 2024-09-13T09:17:38.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531117+0000", 2024-09-13T09:17:38.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6030000000000005e-06 2024-09-13T09:17:38.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.536352+0000", 2024-09-13T09:17:38.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052353260000000002 2024-09-13T09:17:38.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.536628+0000", 2024-09-13T09:17:38.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027556500000000002 2024-09-13T09:17:38.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.536981+0000", 2024-09-13T09:17:38.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035341700000000002 2024-09-13T09:17:38.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.541700+0000", 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047184100000000001 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.541795+0000", 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5031999999999994e-05 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.541955+0000", 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160056 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542078+0000", 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012296200000000001 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531127+0000", 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.024166916999999, 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011389865000000001, 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.572 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 35 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531127+0000", 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531127+0000", 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531126+0000", 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531274+0000", 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014800800000000001 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531288+0000", 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3867e-05 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531297+0000", 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9330000000000007e-06 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.538727+0000", 2024-09-13T09:17:38.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.007429639 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.538949+0000", 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022249499999999999 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.539160+0000", 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000211079 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542252+0000", 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030918460000000001 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542280+0000", 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7433e-05 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542409+0000", 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012945599999999999 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542517+0000", 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107725 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531307+0000", 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.023986441000002, 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012285895, 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.579 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 36 2024-09-13T09:17:38.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.579 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531307+0000", 2024-09-13T09:17:38.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531307+0000", 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531306+0000", 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531313+0000", 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3130000000000004e-06 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531322+0000", 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.85e-06 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531329+0000", 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9600000000000003e-06 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.540210+0000", 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088810439999999994 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.540477+0000", 2024-09-13T09:17:38.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000267438 2024-09-13T09:17:38.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.540862+0000", 2024-09-13T09:17:38.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038524400000000003 2024-09-13T09:17:38.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543383+0000", 2024-09-13T09:17:38.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002520764 2024-09-13T09:17:38.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543447+0000", 2024-09-13T09:17:38.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4063e-05 2024-09-13T09:17:38.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543538+0000", 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1268999999999996e-05 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543593+0000", 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000002e-05 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531337+0000", 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.023956931000001, 2024-09-13T09:17:38.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013657717, 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531337+0000", 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531337+0000", 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531336+0000", 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531344+0000", 2024-09-13T09:17:38.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5599999999999996e-06 2024-09-13T09:17:38.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531353+0000", 2024-09-13T09:17:38.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8869999999999998e-06 2024-09-13T09:17:38.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531360+0000", 2024-09-13T09:17:38.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9100000000000005e-06 2024-09-13T09:17:38.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542600+0000", 2024-09-13T09:17:38.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011239638999999999 2024-09-13T09:17:38.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542747+0000", 2024-09-13T09:17:38.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014643499999999999 2024-09-13T09:17:38.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542947+0000", 2024-09-13T09:17:38.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000200149 2024-09-13T09:17:38.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543918+0000", 2024-09-13T09:17:38.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00097078599999999996 2024-09-13T09:17:38.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.544760+0000", 2024-09-13T09:17:38.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00084245000000000001 2024-09-13T09:17:38.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.544898+0000", 2024-09-13T09:17:38.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013783900000000001 2024-09-13T09:17:38.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.544995+0000", 2024-09-13T09:17:38.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6766000000000002e-05 2024-09-13T09:17:38.590 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.631401+0000", 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.923892433999999, 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010376106, 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 86 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631401+0000", 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631401+0000", 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631400+0000", 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631411+0000", 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1029999999999999e-05 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631429+0000", 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7907e-05 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631443+0000", 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4283e-05 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.636777+0000", 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053336190000000004 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.636951+0000", 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174325 2024-09-13T09:17:38.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.637168+0000", 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021682200000000001 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641519+0000", 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0043510759999999997 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641638+0000", 2024-09-13T09:17:38.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118573 2024-09-13T09:17:38.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641721+0000", 2024-09-13T09:17:38.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2803e-05 2024-09-13T09:17:38.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641777+0000", 2024-09-13T09:17:38.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6771999999999998e-05 2024-09-13T09:17:38.596 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.597 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.597 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.597 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.631461+0000", 2024-09-13T09:17:38.597 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.923832960999999, 2024-09-13T09:17:38.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010750297000000001, 2024-09-13T09:17:38.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 87 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631461+0000", 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631461+0000", 2024-09-13T09:17:38.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631460+0000", 2024-09-13T09:17:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631618+0000", 2024-09-13T09:17:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015849899999999999 2024-09-13T09:17:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631641+0000", 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2912999999999999e-05 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631657+0000", 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6305999999999999e-05 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.638673+0000", 2024-09-13T09:17:38.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0070156300000000001 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.638802+0000", 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129049 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.639028+0000", 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022574100000000001 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641921+0000", 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002893648 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641947+0000", 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5307e-05 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642120+0000", 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017344499999999999 2024-09-13T09:17:38.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642211+0000", 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0876000000000002e-05 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.631676+0000", 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.923617526000001, 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011331733, 2024-09-13T09:17:38.603 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 88 2024-09-13T09:17:38.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.604 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631676+0000", 2024-09-13T09:17:38.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631676+0000", 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631675+0000", 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631687+0000", 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2163e-05 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631705+0000", 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7876999999999998e-05 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631722+0000", 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.685e-05 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.640114+0000", 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083919979999999995 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.640389+0000", 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027471800000000002 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.640788+0000", 2024-09-13T09:17:38.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00039957000000000002 2024-09-13T09:17:38.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642474+0000", 2024-09-13T09:17:38.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001686014 2024-09-13T09:17:38.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642883+0000", 2024-09-13T09:17:38.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040869700000000002 2024-09-13T09:17:38.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642957+0000", 2024-09-13T09:17:38.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4239000000000005e-05 2024-09-13T09:17:38.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.643008+0000", 2024-09-13T09:17:38.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0917000000000001e-05 2024-09-13T09:17:38.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.610 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.610 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.610 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.797962+0000", 2024-09-13T09:17:38.610 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.757331553, 2024-09-13T09:17:38.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01079161, 2024-09-13T09:17:38.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 110 2024-09-13T09:17:38.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.611 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797962+0000", 2024-09-13T09:17:38.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797962+0000", 2024-09-13T09:17:38.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797960+0000", 2024-09-13T09:17:38.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:38.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797974+0000", 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4545999999999999e-05 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797999+0000", 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4573e-05 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798017+0000", 2024-09-13T09:17:38.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7940000000000001e-05 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.804966+0000", 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069496469999999998 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.805091+0000", 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012414199999999999 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.805276+0000", 2024-09-13T09:17:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018536799999999999 2024-09-13T09:17:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808455+0000", 2024-09-13T09:17:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00317914 2024-09-13T09:17:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808580+0000", 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012457500000000001 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808654+0000", 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4373000000000001e-05 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808754+0000", 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9869000000000002e-05 2024-09-13T09:17:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798038+0000", 2024-09-13T09:17:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.757255717, 2024-09-13T09:17:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012627325999999999, 2024-09-13T09:17:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 111 2024-09-13T09:17:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798038+0000", 2024-09-13T09:17:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798038+0000", 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798037+0000", 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798048+0000", 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1194e-05 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798074+0000", 2024-09-13T09:17:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5516000000000001e-05 2024-09-13T09:17:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798088+0000", 2024-09-13T09:17:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4663e-05 2024-09-13T09:17:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.806212+0000", 2024-09-13T09:17:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081233810000000007 2024-09-13T09:17:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.806328+0000", 2024-09-13T09:17:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011605200000000001 2024-09-13T09:17:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.806514+0000", 2024-09-13T09:17:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018678299999999999 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808565+0000", 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020504439999999998 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810494+0000", 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019292109999999999 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810569+0000", 2024-09-13T09:17:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4436000000000004e-05 2024-09-13T09:17:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810665+0000", 2024-09-13T09:17:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6842999999999994e-05 2024-09-13T09:17:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798108+0000", 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.757186091000001, 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014039507999999999, 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798108+0000", 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798108+0000", 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798107+0000", 2024-09-13T09:17:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798267+0000", 2024-09-13T09:17:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160116 2024-09-13T09:17:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798294+0000", 2024-09-13T09:17:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6923e-05 2024-09-13T09:17:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798317+0000", 2024-09-13T09:17:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3693000000000002e-05 2024-09-13T09:17:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.807303+0000", 2024-09-13T09:17:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089853680000000005 2024-09-13T09:17:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.807428+0000", 2024-09-13T09:17:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012510200000000001 2024-09-13T09:17:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.807616+0000", 2024-09-13T09:17:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188145 2024-09-13T09:17:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810479+0000", 2024-09-13T09:17:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028628949999999998 2024-09-13T09:17:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811969+0000", 2024-09-13T09:17:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001489661 2024-09-13T09:17:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812047+0000", 2024-09-13T09:17:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8296999999999999e-05 2024-09-13T09:17:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812147+0000", 2024-09-13T09:17:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100375 2024-09-13T09:17:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798339+0000", 2024-09-13T09:17:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.756954492999999, 2024-09-13T09:17:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013996181999999999, 2024-09-13T09:17:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113 2024-09-13T09:17:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798339+0000", 2024-09-13T09:17:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798339+0000", 2024-09-13T09:17:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798338+0000", 2024-09-13T09:17:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798357+0000", 2024-09-13T09:17:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.897e-05 2024-09-13T09:17:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798404+0000", 2024-09-13T09:17:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7119000000000001e-05 2024-09-13T09:17:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798421+0000", 2024-09-13T09:17:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7187e-05 2024-09-13T09:17:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808797+0000", 2024-09-13T09:17:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010375880000000001 2024-09-13T09:17:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808915+0000", 2024-09-13T09:17:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011846200000000001 2024-09-13T09:17:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809099+0000", 2024-09-13T09:17:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183313 2024-09-13T09:17:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811848+0000", 2024-09-13T09:17:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027493439999999999 2024-09-13T09:17:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812174+0000", 2024-09-13T09:17:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032562200000000002 2024-09-13T09:17:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812244+0000", 2024-09-13T09:17:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0902000000000007e-05 2024-09-13T09:17:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812335+0000", 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0995999999999994e-05 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798442+0000", 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.756851733000001, 2024-09-13T09:17:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015723348000000002, 2024-09-13T09:17:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114 2024-09-13T09:17:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798442+0000", 2024-09-13T09:17:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798442+0000", 2024-09-13T09:17:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798441+0000", 2024-09-13T09:17:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798453+0000", 2024-09-13T09:17:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2784e-05 2024-09-13T09:17:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798476+0000", 2024-09-13T09:17:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2875999999999999e-05 2024-09-13T09:17:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798490+0000", 2024-09-13T09:17:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3482999999999999e-05 2024-09-13T09:17:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809646+0000", 2024-09-13T09:17:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011155847 2024-09-13T09:17:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809754+0000", 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108256 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809965+0000", 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000210659 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811953+0000", 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019889439999999999 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813962+0000", 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020080850000000002 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814040+0000", 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8184999999999998e-05 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814165+0000", 2024-09-13T09:17:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012563299999999999 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798506+0000", 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.756787364000001, 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015834198000000001, 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 115 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798506+0000", 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798506+0000", 2024-09-13T09:17:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798505+0000", 2024-09-13T09:17:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798658+0000", 2024-09-13T09:17:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000153005 2024-09-13T09:17:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798681+0000", 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2634e-05 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798697+0000", 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5696e-05 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810746+0000", 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012049331 2024-09-13T09:17:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810873+0000", 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012754899999999999 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811051+0000", 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000177252 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813948+0000", 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028975239999999998 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814188+0000", 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000239724 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814255+0000", 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7372999999999993e-05 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814341+0000", 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5270000000000002e-05 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798716+0000", 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.756577519, 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015827386999999998, 2024-09-13T09:17:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116 2024-09-13T09:17:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798716+0000", 2024-09-13T09:17:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798716+0000", 2024-09-13T09:17:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798715+0000", 2024-09-13T09:17:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798727+0000", 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1800000000000001e-05 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798746+0000", 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8992999999999999e-05 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798765+0000", 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9049e-05 2024-09-13T09:17:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812383+0000", 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013618531999999999 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812498+0000", 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115056 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812682+0000", 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018366899999999999 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814367+0000", 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00168501 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814421+0000", 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4199000000000001e-05 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814492+0000", 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0502999999999999e-05 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814544+0000", 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2039000000000001e-05 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798781+0000", 2024-09-13T09:17:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.756512633, 2024-09-13T09:17:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016163997999999999, 2024-09-13T09:17:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 117 2024-09-13T09:17:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798781+0000", 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798781+0000", 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798780+0000", 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798795+0000", 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4574e-05 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798813+0000", 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8536e-05 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798829+0000", 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5866999999999999e-05 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813236+0000", 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014407229000000001 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813350+0000", 2024-09-13T09:17:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011376500000000001 2024-09-13T09:17:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813530+0000", 2024-09-13T09:17:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179619 2024-09-13T09:17:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814654+0000", 2024-09-13T09:17:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011244339999999999 2024-09-13T09:17:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814768+0000", 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011397599999999999 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814845+0000", 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7299999999999995e-05 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814945+0000", 2024-09-13T09:17:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9875000000000002e-05 2024-09-13T09:17:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:12.288452+0000", 2024-09-13T09:17:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.266841948, 2024-09-13T09:17:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010418200000000001, 2024-09-13T09:17:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140 2024-09-13T09:17:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288452+0000", 2024-09-13T09:17:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288452+0000", 2024-09-13T09:17:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288451+0000", 2024-09-13T09:17:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288508+0000", 2024-09-13T09:17:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6706999999999998e-05 2024-09-13T09:17:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288517+0000", 2024-09-13T09:17:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8339999999999994e-06 2024-09-13T09:17:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288523+0000", 2024-09-13T09:17:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5419999999999996e-06 2024-09-13T09:17:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.291720+0000", 2024-09-13T09:17:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031973499999999998 2024-09-13T09:17:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.291765+0000", 2024-09-13T09:17:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.443e-05 2024-09-13T09:17:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.291834+0000", 2024-09-13T09:17:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9053000000000004e-05 2024-09-13T09:17:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.293879+0000", 2024-09-13T09:17:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020453619999999998 2024-09-13T09:17:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.298804+0000", 2024-09-13T09:17:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049248069999999998 2024-09-13T09:17:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.298831+0000", 2024-09-13T09:17:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7436e-05 2024-09-13T09:17:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.298870+0000", 2024-09-13T09:17:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8549e-05 2024-09-13T09:17:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:12.288530+0000", 2024-09-13T09:17:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.266763991000001, 2024-09-13T09:17:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011049003999999999, 2024-09-13T09:17:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 141 2024-09-13T09:17:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288530+0000", 2024-09-13T09:17:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288530+0000", 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288529+0000", 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288532+0000", 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288536+0000", 2024-09-13T09:17:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8480000000000003e-06 2024-09-13T09:17:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288539+0000", 2024-09-13T09:17:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0479999999999999e-06 2024-09-13T09:17:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.292188+0000", 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036491639999999999 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.292357+0000", 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016818200000000001 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.292492+0000", 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013562099999999999 2024-09-13T09:17:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.293885+0000", 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001393044 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.299519+0000", 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056339670000000001 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.299547+0000", 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7928000000000001e-05 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.299579+0000", 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1547999999999997e-05 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.515193632999999, 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010963133, 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 158 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040255+0000", 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015546599999999999 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040268+0000", 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2860000000000001e-05 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040276+0000", 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5399999999999996e-06 2024-09-13T09:17:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.047322+0000", 2024-09-13T09:17:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0070454380000000002 2024-09-13T09:17:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.047443+0000", 2024-09-13T09:17:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012126 2024-09-13T09:17:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.047631+0000", 2024-09-13T09:17:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000187558 2024-09-13T09:17:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050775+0000", 2024-09-13T09:17:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031440000000000001 2024-09-13T09:17:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050797+0000", 2024-09-13T09:17:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2730000000000001e-05 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050928+0000", 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013016500000000001 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051063+0000", 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013571300000000001 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:17:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:13.040286+0000", 2024-09-13T09:17:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.515007471000001, 2024-09-13T09:17:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011356279, 2024-09-13T09:17:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 159 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040286+0000", 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040286+0000", 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040285+0000", 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040296+0000", 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0473e-05 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040305+0000", 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0070000000000003e-06 2024-09-13T09:17:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040315+0000", 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8500000000000006e-06 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.049739+0000", 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094246899999999995 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.049892+0000", 2024-09-13T09:17:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015284900000000001 2024-09-13T09:17:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050097+0000", 2024-09-13T09:17:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020424500000000001 2024-09-13T09:17:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051387+0000", 2024-09-13T09:17:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012901970000000001 2024-09-13T09:17:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051511+0000", 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124039 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051589+0000", 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.873e-05 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051643+0000", 2024-09-13T09:17:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3102000000000001e-05 2024-09-13T09:17:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:38.691 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:38.693 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:38.686+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:38.694 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:17:38.694 INFO:teuthology.orchestra.run.smithi032.stdout:792 writesame 0xe13b8b2 thru 0xe13bd8a data_size 0x1(0x4d9 bytes) 2024-09-13T09:17:38.694 INFO:teuthology.orchestra.run.smithi032.stdout:795 write 0x2c776d0 thru 0x2c77bf9 (0x52a bytes) 2024-09-13T09:17:38.694 INFO:teuthology.orchestra.run.smithi032.stdout:796 punch from 0x2f19e90 to 0x2f1e702, (0x4872 bytes) 2024-09-13T09:17:38.695 INFO:teuthology.orchestra.run.smithi032.stdout:797 trunc from 0xe13bd8b to 0x5ddcb45 2024-09-13T09:17:38.695 INFO:teuthology.orchestra.run.smithi032.stdout:798 writesame 0xcc88df5 thru 0xcc8cdd6 data_size 0x25(0x3fe2 bytes) 2024-09-13T09:17:38.695 INFO:teuthology.orchestra.run.smithi032.stdout:799 read 0x4382bb3 thru 0x438bd0f (0x915d bytes) 2024-09-13T09:17:38.695 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xebd584e 2024-09-13T09:17:38.695 INFO:teuthology.orchestra.run.smithi032.stdout:801 trunc from 0xcc8cdd7 to 0xebd584e 2024-09-13T09:17:38.695 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:38.695 INFO:teuthology.orchestra.run.smithi032.stdout:805 writesame 0xb315b8c thru 0xb3254ad data_size 0x2(0xf922 bytes) 2024-09-13T09:17:38.737 INFO:teuthology.orchestra.run.smithi032.stdout:806 trunc from 0xebd584e to 0xbbf16a2 2024-09-13T09:17:38.748 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:17:38.748 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:17:38.749 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-13T09:17:38.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-13T09:17:38.751 INFO:teuthology.orchestra.run.smithi032.stdout:807 write 0x87af2c1 thru 0x87b4325 (0x5065 bytes) 2024-09-13T09:17:38.764 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:17:38.774 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:38.774 INFO:teuthology.orchestra.run.smithi032.stdout:811 punch from 0xac008ad to 0xac0cd46, (0xc499 bytes) 2024-09-13T09:17:38.774 INFO:teuthology.orchestra.run.smithi032.stdout:812 clone 4 order 18 su 131072 sc 15 2024-09-13T09:17:38.830 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:38.828+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:38.838 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:38.838 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:38.838 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:38.838 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:38.853 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:17:38.894 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:38.890+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:38.995 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:38.990+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:39.016 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:39.016 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:39.016 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:39.016 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:39.016 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:39.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:17:39.061 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-09-13T09:17:39.056+0000 7f1d86e9f640 -1 mon.a@0(leader).osd e32 definitely_dead 0 2024-09-13T09:17:39.096 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:39.092+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:39.161 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:39.161 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:39.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:39.161 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:39.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.161 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-13T09:17:39.161 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:11.579976+0000", 2024-09-13T09:17:39.161 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.579858169000001, 2024-09-13T09:17:39.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002347348, 2024-09-13T09:17:39.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.162 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.579976+0000", 2024-09-13T09:17:39.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.579976+0000", 2024-09-13T09:17:39.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.579974+0000", 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.579994+0000", 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0083000000000001e-05 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.580032+0000", 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7741999999999999e-05 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.580069+0000", 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7177e-05 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.580185+0000", 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011610199999999999 2024-09-13T09:17:39.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.580223+0000", 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7540000000000003e-05 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.582239+0000", 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020165679999999998 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.582324+0000", 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4369000000000005e-05 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:57 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:11.586713+0000", 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.573120981999999, 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0023464390000000001, 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.166 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586713+0000", 2024-09-13T09:17:39.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586713+0000", 2024-09-13T09:17:39.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586712+0000", 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586767+0000", 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5726999999999997e-05 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586797+0000", 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9266999999999999e-05 2024-09-13T09:17:39.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586811+0000", 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4679e-05 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586916+0000", 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010517 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.586952+0000", 2024-09-13T09:17:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5228999999999998e-05 2024-09-13T09:17:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.588980+0000", 2024-09-13T09:17:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020282009999999999 2024-09-13T09:17:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.589060+0000", 2024-09-13T09:17:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0023000000000003e-05 2024-09-13T09:17:39.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:11.608699+0000", 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.551135498000001, 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025443969999999999, 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608699+0000", 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608699+0000", 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608697+0000", 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:39.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608757+0000", 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9552999999999997e-05 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608793+0000", 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5902999999999998e-05 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608812+0000", 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9233e-05 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.608987+0000", 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017490600000000001 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.609022+0000", 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4946000000000001e-05 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.611165+0000", 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021434539999999999 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:11.611243+0000", 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8242000000000001e-05 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:129 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'87, mlcod=16'85)", 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.288944+0000", 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.870890661000001, 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0048963449999999999, 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288944+0000", 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288944+0000", 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288942+0000", 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288960+0000", 2024-09-13T09:17:39.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8077e-05 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288989+0000", 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9476e-05 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289003+0000", 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3046e-05 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289103+0000", 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010054 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289137+0000", 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3575999999999998e-05 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293750+0000", 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046137610000000001 2024-09-13T09:17:39.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293840+0000", 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9698999999999996e-05 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:130 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'88, mlcod=16'85)", 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.289290+0000", 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.870544556999999, 2024-09-13T09:17:39.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046447700000000003, 2024-09-13T09:17:39.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.181 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.181 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289290+0000", 2024-09-13T09:17:39.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289290+0000", 2024-09-13T09:17:39.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289289+0000", 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289299+0000", 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6430000000000007e-06 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289315+0000", 2024-09-13T09:17:39.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.609e-05 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289324+0000", 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3430000000000002e-06 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289718+0000", 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039453699999999998 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289792+0000", 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3786000000000001e-05 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293848+0000", 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0040556020000000002 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293935+0000", 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6749000000000006e-05 2024-09-13T09:17:39.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:131 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.289777+0000", 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.870057708000001, 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005922531, 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289777+0000", 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289777+0000", 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289775+0000", 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289961+0000", 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018587200000000001 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289993+0000", 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2007e-05 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290016+0000", 2024-09-13T09:17:39.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3025999999999999e-05 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290280+0000", 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026384800000000001 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290318+0000", 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7945999999999999e-05 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295642+0000", 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0053246159999999999 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295699+0000", 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6898999999999999e-05 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:132 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-13T09:17:39.190 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.290371+0000", 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.869463259, 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0053750650000000001, 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290371+0000", 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290371+0000", 2024-09-13T09:17:39.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290369+0000", 2024-09-13T09:17:39.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:39.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290384+0000", 2024-09-13T09:17:39.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5133999999999999e-05 2024-09-13T09:17:39.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290419+0000", 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4555999999999999e-05 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290437+0000", 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8343e-05 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291376+0000", 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00093908299999999995 2024-09-13T09:17:39.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291410+0000", 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3883000000000001e-05 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295705+0000", 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042944790000000004 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295746+0000", 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1496999999999999e-05 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:133 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.290748+0000", 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.869086238000001, 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0050401270000000001, 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290748+0000", 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290748+0000", 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290747+0000", 2024-09-13T09:17:39.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:39.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290761+0000", 2024-09-13T09:17:39.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4236000000000001e-05 2024-09-13T09:17:39.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290788+0000", 2024-09-13T09:17:39.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6826999999999999e-05 2024-09-13T09:17:39.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290804+0000", 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5798999999999999e-05 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291981+0000", 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011771080000000001 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292015+0000", 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4693000000000001e-05 2024-09-13T09:17:39.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295750+0000", 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037345780000000001 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295788+0000", 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:134 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.291161+0000", 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.868673512000001, 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064130539999999996, 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291161+0000", 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291161+0000", 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291159+0000", 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291257+0000", 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7498999999999996e-05 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291284+0000", 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291305+0000", 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1773e-05 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292753+0000", 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014473719999999999 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292786+0000", 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3370000000000001e-05 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297518+0000", 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0047318569999999999 2024-09-13T09:17:39.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297574+0000", 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6026000000000002e-05 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:135 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'85)", 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.291641+0000", 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.868193412, 2024-09-13T09:17:39.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0059934710000000002, 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291641+0000", 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291641+0000", 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291639+0000", 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291653+0000", 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3977e-05 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291684+0000", 2024-09-13T09:17:39.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0298999999999999e-05 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291701+0000", 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7847e-05 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294015+0000", 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002313402 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294047+0000", 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2063000000000001e-05 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297582+0000", 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035351430000000001 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297634+0000", 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2460000000000003e-05 2024-09-13T09:17:39.208 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'85)", 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.292054+0000", 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.867780037999999, 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0073146829999999998, 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292054+0000", 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292054+0000", 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292053+0000", 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292067+0000", 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.411e-05 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292092+0000", 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5173000000000001e-05 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292111+0000", 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8963000000000001e-05 2024-09-13T09:17:39.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294704+0000", 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025933800000000002 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294735+0000", 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0366e-05 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299316+0000", 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004581412 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299369+0000", 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3016000000000002e-05 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:137 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'85)", 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.292515+0000", 2024-09-13T09:17:39.213 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.867319828999999, 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068905700000000004, 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292515+0000", 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292515+0000", 2024-09-13T09:17:39.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292513+0000", 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292586+0000", 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3139999999999994e-05 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292618+0000", 2024-09-13T09:17:39.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1878999999999998e-05 2024-09-13T09:17:39.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292639+0000", 2024-09-13T09:17:39.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0752999999999999e-05 2024-09-13T09:17:39.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295880+0000", 2024-09-13T09:17:39.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0032411889999999998 2024-09-13T09:17:39.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295927+0000", 2024-09-13T09:17:39.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7175999999999997e-05 2024-09-13T09:17:39.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299375+0000", 2024-09-13T09:17:39.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0034476400000000001 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299405+0000", 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0642999999999998e-05 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:138 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'85)", 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.292944+0000", 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.866890195, 2024-09-13T09:17:39.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064997630000000004, 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292944+0000", 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292944+0000", 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292943+0000", 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292957+0000", 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4610000000000001e-05 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292982+0000", 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5003000000000001e-05 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293000+0000", 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8142999999999999e-05 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.296939+0000", 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0039383700000000001 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.296972+0000", 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2716999999999999e-05 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299412+0000", 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024409340000000001 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299444+0000", 2024-09-13T09:17:39.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.154e-05 2024-09-13T09:17:39.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.223 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.223 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:139 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'85)", 2024-09-13T09:17:39.223 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.293238+0000", 2024-09-13T09:17:39.223 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.866596310999999, 2024-09-13T09:17:39.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062557749999999999, 2024-09-13T09:17:39.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.223 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.223 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293238+0000", 2024-09-13T09:17:39.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293238+0000", 2024-09-13T09:17:39.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293237+0000", 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293250+0000", 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.379e-05 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293276+0000", 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6126e-05 2024-09-13T09:17:39.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293295+0000", 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8904000000000001e-05 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297711+0000", 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044153789999999997 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297745+0000", 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4499999999999998e-05 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299450+0000", 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00170436 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299494+0000", 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4302999999999999e-05 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'85)", 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.293606+0000", 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.866228033999999, 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066098429999999998, 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293606+0000", 2024-09-13T09:17:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293606+0000", 2024-09-13T09:17:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293605+0000", 2024-09-13T09:17:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293707+0000", 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010204600000000001 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293732+0000", 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5429999999999999e-05 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293749+0000", 2024-09-13T09:17:39.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6390000000000001e-05 2024-09-13T09:17:39.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.298325+0000", 2024-09-13T09:17:39.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045769440000000003 2024-09-13T09:17:39.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.298370+0000", 2024-09-13T09:17:39.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4187000000000002e-05 2024-09-13T09:17:39.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300157+0000", 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017874690000000001 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300216+0000", 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9067000000000002e-05 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:141 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'85)", 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.294217+0000", 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.865617798999999, 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067059559999999999, 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294217+0000", 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294217+0000", 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294215+0000", 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294229+0000", 2024-09-13T09:17:39.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3974e-05 2024-09-13T09:17:39.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294261+0000", 2024-09-13T09:17:39.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1903000000000003e-05 2024-09-13T09:17:39.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294280+0000", 2024-09-13T09:17:39.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9589000000000002e-05 2024-09-13T09:17:39.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299531+0000", 2024-09-13T09:17:39.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0052504029999999998 2024-09-13T09:17:39.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299567+0000", 2024-09-13T09:17:39.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6427000000000002e-05 2024-09-13T09:17:39.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300868+0000", 2024-09-13T09:17:39.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013010470000000001 2024-09-13T09:17:39.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300923+0000", 2024-09-13T09:17:39.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4203000000000003e-05 2024-09-13T09:17:39.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.237 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:142 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'100, mlcod=16'98)", 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.304015+0000", 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.855819401000002, 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024443709999999999, 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304015+0000", 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304015+0000", 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304013+0000", 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304261+0000", 2024-09-13T09:17:39.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024744099999999999 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304284+0000", 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3370000000000002e-05 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304298+0000", 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3502999999999999e-05 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304384+0000", 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6650000000000006e-05 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304424+0000", 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9941999999999998e-05 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.306398+0000", 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019733390000000002 2024-09-13T09:17:39.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.306459+0000", 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1869e-05 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:143 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.304318+0000", 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.855516753, 2024-09-13T09:17:39.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029677929999999998, 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304318+0000", 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304318+0000", 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304317+0000", 2024-09-13T09:17:39.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:39.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304426+0000", 2024-09-13T09:17:39.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109209 2024-09-13T09:17:39.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304450+0000", 2024-09-13T09:17:39.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4090000000000001e-05 2024-09-13T09:17:39.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304462+0000", 2024-09-13T09:17:39.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2480000000000001e-05 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.305073+0000", 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00061091499999999996 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.305112+0000", 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8523999999999999e-05 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307197+0000", 2024-09-13T09:17:39.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020850439999999999 2024-09-13T09:17:39.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307285+0000", 2024-09-13T09:17:39.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8684999999999995e-05 2024-09-13T09:17:39.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.247 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:144 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.304477+0000", 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.855357474000002, 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029682900000000002, 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304477+0000", 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304477+0000", 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304476+0000", 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304603+0000", 2024-09-13T09:17:39.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012634500000000001 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304620+0000", 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7450000000000001e-05 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304629+0000", 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4599999999999992e-06 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.305775+0000", 2024-09-13T09:17:39.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001145725 2024-09-13T09:17:39.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.305808+0000", 2024-09-13T09:17:39.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2932999999999998e-05 2024-09-13T09:17:39.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307394+0000", 2024-09-13T09:17:39.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015858269999999999 2024-09-13T09:17:39.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307445+0000", 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1329999999999998e-05 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:145 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.304643+0000", 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.855191816000001, 2024-09-13T09:17:39.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0032891209999999999, 2024-09-13T09:17:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304643+0000", 2024-09-13T09:17:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304643+0000", 2024-09-13T09:17:39.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304642+0000", 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304743+0000", 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101049 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304775+0000", 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1973e-05 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.304783+0000", 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.306544+0000", 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017610659999999999 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.306577+0000", 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3149999999999999e-05 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307851+0000", 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001273843 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.307932+0000", 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0465999999999998e-05 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.256 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.257 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.257 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.257 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.257 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:39.258 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:17:39.259 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:39.254+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:39.359 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:39.358+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:39.432 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:39.432 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:39.432 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:39.433 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:39.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:17:39.460 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:39.454+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:39.561 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:39.556+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:39.566 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:39.567 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:39.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:17:39.666 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:39.664+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:39.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:17:39.747 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:39.747 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:39.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:39.747 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:39.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.747 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:995 3.1a e28/22 3:5db8ce44:::rbd_data.1185192b47c4.000000000000001e:head v 28'10, mlcod=27'8)", 2024-09-13T09:17:39.747 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.173001+0000", 2024-09-13T09:17:39.747 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.5717463770000002, 2024-09-13T09:17:39.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030959490000000002, 2024-09-13T09:17:39.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.748 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.748 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.173001+0000", 2024-09-13T09:17:39.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.173001+0000", 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.173000+0000", 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.174300+0000", 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001300278 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.174313+0000", 2024-09-13T09:17:39.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.236e-05 2024-09-13T09:17:39.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.174320+0000", 2024-09-13T09:17:39.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9999999999999999e-06 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.174422+0000", 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101668 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.174462+0000", 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0593000000000003e-05 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.176028+0000", 2024-09-13T09:17:39.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015657749999999999 2024-09-13T09:17:39.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.176097+0000", 2024-09-13T09:17:39.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9091999999999995e-05 2024-09-13T09:17:39.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.752 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1019 3.19 3:9c410407:::rbd_data.1185192b47c4.0000000000000039:head [set-alloc-hint object_size 4194304 write_size 4194304,write 425877~24501] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.253485+0000", 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.4912625080000002, 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033370040000000002, 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1019 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253485+0000", 2024-09-13T09:17:39.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253485+0000", 2024-09-13T09:17:39.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253481+0000", 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253542+0000", 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0723000000000001e-05 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253583+0000", 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0822999999999999e-05 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253600+0000", 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6883000000000001e-05 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253712+0000", 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011202 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.253993+0000", 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028126399999999998 2024-09-13T09:17:39.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.254252+0000", 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025908799999999998 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.256125+0000", 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001872439 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.256673+0000", 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054806199999999998 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.256758+0000", 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5439999999999995e-05 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.256822+0000", 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3859e-05 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1035 3.0 3:02e5ffb1:::rbd_data.1185192b47c4.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 3273017~65270] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.298987+0000", 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.4457603560000001, 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0034987960000000002, 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1035 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.759 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.298987+0000", 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.298987+0000", 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.298985+0000", 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.299088+0000", 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103199 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.299121+0000", 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2546999999999999e-05 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.299145+0000", 2024-09-13T09:17:39.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4000000000000001e-05 2024-09-13T09:17:39.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.299274+0000", 2024-09-13T09:17:39.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012890500000000001 2024-09-13T09:17:39.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.299530+0000", 2024-09-13T09:17:39.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000256035 2024-09-13T09:17:39.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:39.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.299845+0000", 2024-09-13T09:17:39.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031564000000000001 2024-09-13T09:17:39.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.302085+0000", 2024-09-13T09:17:39.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022398029999999998 2024-09-13T09:17:39.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.302341+0000", 2024-09-13T09:17:39.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025572799999999999 2024-09-13T09:17:39.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.302427+0000", 2024-09-13T09:17:39.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6459999999999996e-05 2024-09-13T09:17:39.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.302486+0000", 2024-09-13T09:17:39.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8649e-05 2024-09-13T09:17:39.765 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.765 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.765 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.765 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1078 3.4 3:2221ff39:::rbd_data.1185192b47c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 3023129~32394] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:39.765 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.397885+0000", 2024-09-13T09:17:39.765 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.346862099, 2024-09-13T09:17:39.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035396859999999998, 2024-09-13T09:17:39.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.765 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1078 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.397885+0000", 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.397885+0000", 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.397882+0000", 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.397924+0000", 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2002999999999997e-05 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.397964+0000", 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9829000000000001e-05 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.397981+0000", 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.649e-05 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.398081+0000", 2024-09-13T09:17:39.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100373 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.398333+0000", 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025160800000000001 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.398696+0000", 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036374699999999999 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.400471+0000", 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001775113 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.401269+0000", 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00079757700000000005 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.401357+0000", 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8518999999999997e-05 2024-09-13T09:17:39.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.401425+0000", 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7500000000000001e-05 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1090 3.6 3:6098d441:::rbd_data.1185192b47c4.0000000000000036:head [set-alloc-hint object_size 4194304 write_size 4194304,write 784719~35429] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.421825+0000", 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.3229228410000005, 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003158639, 2024-09-13T09:17:39.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1090 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.421825+0000", 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.421825+0000", 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.421822+0000", 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.421862+0000", 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9956000000000002e-05 2024-09-13T09:17:39.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.421895+0000", 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3869999999999999e-05 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.421911+0000", 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5356e-05 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.422032+0000", 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000121119 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.422256+0000", 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022440599999999999 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.422476+0000", 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021970799999999999 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.424202+0000", 2024-09-13T09:17:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017259129999999999 2024-09-13T09:17:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.424833+0000", 2024-09-13T09:17:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00063072200000000001 2024-09-13T09:17:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.424921+0000", 2024-09-13T09:17:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8468999999999996e-05 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.424983+0000", 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2063000000000005e-05 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1123 3.0 3:0731d9f1:::rbd_data.1185192b47c4.0000000000000032:head [set-alloc-hint object_size 4194304 write_size 4194304,write 793280~26123] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.501430+0000", 2024-09-13T09:17:39.777 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.2433174280000001, 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0034807800000000002, 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1123 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501430+0000", 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501430+0000", 2024-09-13T09:17:39.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501427+0000", 2024-09-13T09:17:39.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501460+0000", 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2972000000000002e-05 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501492+0000", 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1937000000000003e-05 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501510+0000", 2024-09-13T09:17:39.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7733000000000001e-05 2024-09-13T09:17:39.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501609+0000", 2024-09-13T09:17:39.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9445999999999996e-05 2024-09-13T09:17:39.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.501844+0000", 2024-09-13T09:17:39.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023497500000000001 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.502071+0000", 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022678499999999999 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.503823+0000", 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017522690000000001 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.504689+0000", 2024-09-13T09:17:39.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00086600700000000002 2024-09-13T09:17:39.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.504848+0000", 2024-09-13T09:17:39.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000158632 2024-09-13T09:17:39.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.504911+0000", 2024-09-13T09:17:39.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2626999999999995e-05 2024-09-13T09:17:39.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.784 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.784 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1126 3.0 3:02e5ffb1:::rbd_data.1185192b47c4.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:39.784 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.543493+0000", 2024-09-13T09:17:39.784 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.2012545130000003, 2024-09-13T09:17:39.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029711500000000001, 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1126 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543493+0000", 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543493+0000", 2024-09-13T09:17:39.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543490+0000", 2024-09-13T09:17:39.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:39.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543507+0000", 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6773e-05 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543558+0000", 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1069999999999997e-05 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543576+0000", 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8850000000000001e-05 2024-09-13T09:17:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543670+0000", 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3103000000000006e-05 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.543813+0000", 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000143862 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:39.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.544001+0000", 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187608 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.545551+0000", 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001550095 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.546305+0000", 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075378700000000001 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.546406+0000", 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101523 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.546464+0000", 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7618999999999998e-05 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1137 3.0 3:02cec37e:::rbd_data.1185192b47c4.0000000000000019:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.547808+0000", 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1969395370000004, 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003778712, 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1137 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.547808+0000", 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.547808+0000", 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.547805+0000", 2024-09-13T09:17:39.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:39.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.547821+0000", 2024-09-13T09:17:39.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6093e-05 2024-09-13T09:17:39.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.547881+0000", 2024-09-13T09:17:39.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9256000000000003e-05 2024-09-13T09:17:39.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.547905+0000", 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4057e-05 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.548025+0000", 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120442 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.548260+0000", 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023489100000000001 2024-09-13T09:17:39.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.548534+0000", 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027368500000000002 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551296+0000", 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027617459999999998 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551386+0000", 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0031999999999995e-05 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551498+0000", 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112716 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551587+0000", 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8313000000000006e-05 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.796 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.797 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1145 3.0 3:0731d9f1:::rbd_data.1185192b47c4.0000000000000032:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:39.797 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.550114+0000", 2024-09-13T09:17:39.797 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1946330840000003, 2024-09-13T09:17:39.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036077209999999999, 2024-09-13T09:17:39.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.797 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1145 2024-09-13T09:17:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550114+0000", 2024-09-13T09:17:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550114+0000", 2024-09-13T09:17:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550112+0000", 2024-09-13T09:17:39.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:39.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550124+0000", 2024-09-13T09:17:39.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.19e-05 2024-09-13T09:17:39.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550153+0000", 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8430000000000001e-05 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550191+0000", 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8633000000000001e-05 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550307+0000", 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115779 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550552+0000", 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000245262 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.550763+0000", 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000211101 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.552233+0000", 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014694790000000001 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.553567+0000", 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013338180000000001 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.553632+0000", 2024-09-13T09:17:39.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4900000000000005e-05 2024-09-13T09:17:39.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.553722+0000", 2024-09-13T09:17:39.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0476000000000006e-05 2024-09-13T09:17:39.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.803 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.803 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1156 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:39.803 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.562031+0000", 2024-09-13T09:17:39.804 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1827163159999996, 2024-09-13T09:17:39.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0034290560000000002, 2024-09-13T09:17:39.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1156 2024-09-13T09:17:39.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.804 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562031+0000", 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562031+0000", 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562029+0000", 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:39.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562047+0000", 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8093000000000001e-05 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562080+0000", 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3806000000000001e-05 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562099+0000", 2024-09-13T09:17:39.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8502999999999999e-05 2024-09-13T09:17:39.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562185+0000", 2024-09-13T09:17:39.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5805999999999998e-05 2024-09-13T09:17:39.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562466+0000", 2024-09-13T09:17:39.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028099100000000002 2024-09-13T09:17:39.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562971+0000", 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050485000000000005 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.564647+0000", 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001675887 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.565278+0000", 2024-09-13T09:17:39.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00063119800000000002 2024-09-13T09:17:39.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.565373+0000", 2024-09-13T09:17:39.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5636000000000004e-05 2024-09-13T09:17:39.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.565460+0000", 2024-09-13T09:17:39.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6798999999999993e-05 2024-09-13T09:17:39.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1256 3.19 3:9bf4724a:::rbd_data.1185192b47c4.0000000000000020:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.834936+0000", 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.9098115490000005, 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003591136, 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.811 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1256 2024-09-13T09:17:39.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.811 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834936+0000", 2024-09-13T09:17:39.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834936+0000", 2024-09-13T09:17:39.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834934+0000", 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834945+0000", 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0890000000000001e-05 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834971+0000", 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5919999999999999e-05 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.834983+0000", 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2403e-05 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.835045+0000", 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2323000000000006e-05 2024-09-13T09:17:39.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.835105+0000", 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9369e-05 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.835517+0000", 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00041229599999999998 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.838427+0000", 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029098589999999999 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.838438+0000", 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1052e-05 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.838493+0000", 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4987000000000001e-05 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.838527+0000", 2024-09-13T09:17:39.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3943999999999999e-05 2024-09-13T09:17:39.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.816 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.816 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1263 3.19 3:9bf4724a:::rbd_data.1185192b47c4.0000000000000020:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:39.816 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.878974+0000", 2024-09-13T09:17:39.816 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8657735820000001, 2024-09-13T09:17:39.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030362929999999998, 2024-09-13T09:17:39.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.817 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.817 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.817 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.817 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.817 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1263 2024-09-13T09:17:39.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.817 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.878974+0000", 2024-09-13T09:17:39.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.878974+0000", 2024-09-13T09:17:39.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.878971+0000", 2024-09-13T09:17:39.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:39.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.878986+0000", 2024-09-13T09:17:39.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5699e-05 2024-09-13T09:17:39.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.879025+0000", 2024-09-13T09:17:39.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8617e-05 2024-09-13T09:17:39.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.879041+0000", 2024-09-13T09:17:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6016e-05 2024-09-13T09:17:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.879186+0000", 2024-09-13T09:17:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014511600000000001 2024-09-13T09:17:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.879442+0000", 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025617499999999998 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.879784+0000", 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00034151299999999999 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.881351+0000", 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015671750000000001 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.881844+0000", 2024-09-13T09:17:39.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049293399999999999 2024-09-13T09:17:39.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.881919+0000", 2024-09-13T09:17:39.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4734999999999995e-05 2024-09-13T09:17:39.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.882010+0000", 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1442999999999998e-05 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1268 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.887012+0000", 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8577350910000003, 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031407309999999999, 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.824 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1268 2024-09-13T09:17:39.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.824 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887012+0000", 2024-09-13T09:17:39.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887012+0000", 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887010+0000", 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887025+0000", 2024-09-13T09:17:39.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5667000000000001e-05 2024-09-13T09:17:39.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887068+0000", 2024-09-13T09:17:39.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2895999999999997e-05 2024-09-13T09:17:39.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887091+0000", 2024-09-13T09:17:39.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2796e-05 2024-09-13T09:17:39.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887204+0000", 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011266599999999999 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887321+0000", 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011748600000000001 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887510+0000", 2024-09-13T09:17:39.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018889800000000001 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.889400+0000", 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001889759 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.889939+0000", 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00053940699999999999 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.890056+0000", 2024-09-13T09:17:39.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116816 2024-09-13T09:17:39.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.890153+0000", 2024-09-13T09:17:39.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7013000000000001e-05 2024-09-13T09:17:39.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.829 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1271 3.4 3:2221ff39:::rbd_data.1185192b47c4.0000000000000004:head [stat,truncate 1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.887118+0000", 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8576292519999997, 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002935764, 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1271 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887118+0000", 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887118+0000", 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887117+0000", 2024-09-13T09:17:39.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887126+0000", 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7940000000000008e-06 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887148+0000", 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2402999999999999e-05 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887164+0000", 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5736999999999999e-05 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887252+0000", 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7913999999999999e-05 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887368+0000", 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116239 2024-09-13T09:17:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.887552+0000", 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018406999999999999 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.889459+0000", 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001907249 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.889857+0000", 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039799300000000002 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.889958+0000", 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010056300000000001 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.890054+0000", 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6151999999999997e-05 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:1380 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:head [stat,truncate 1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:34.167107+0000", 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.577640089, 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030987160000000001, 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1380 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167107+0000", 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167107+0000", 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167104+0000", 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167124+0000", 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9826999999999998e-05 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167170+0000", 2024-09-13T09:17:39.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6706000000000003e-05 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167194+0000", 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3156e-05 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167328+0000", 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013484200000000001 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167552+0000", 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022331900000000001 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.167924+0000", 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037255300000000002 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.170038+0000", 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002113967 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.170075+0000", 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6494e-05 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.170153+0000", 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8574999999999999e-05 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.170206+0000", 2024-09-13T09:17:39.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2744000000000002e-05 2024-09-13T09:17:39.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.842 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.842 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:2329 3.0 3:038e1668:::rbd_data.1185be6cd1eb.0000000000000006:head [stat,truncate 1048576] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:39.842 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:37.872740+0000", 2024-09-13T09:17:39.842 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.8720077550000001, 2024-09-13T09:17:39.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029806609999999999, 2024-09-13T09:17:39.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.842 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2329 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.872740+0000", 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.872740+0000", 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.872737+0000", 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.872750+0000", 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3317e-05 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.872781+0000", 2024-09-13T09:17:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0522999999999999e-05 2024-09-13T09:17:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.872796+0000", 2024-09-13T09:17:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4633e-05 2024-09-13T09:17:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.872892+0000", 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6028999999999998e-05 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.873033+0000", 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141886 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.873344+0000", 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031087999999999998 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.875621+0000", 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022764259999999998 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.875629+0000", 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4700000000000002e-06 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.875678+0000", 2024-09-13T09:17:39.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8989000000000001e-05 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:37.875720+0000", 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2048000000000001e-05 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:2896 3.4 3:22b919ae:::rbd_data.118591b29fad.00000000000000a3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:38.298110+0000", 2024-09-13T09:17:39.848 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.4466372949999999, 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002974726, 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2896 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.298110+0000", 2024-09-13T09:17:39.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.298110+0000", 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.298107+0000", 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.298140+0000", 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2595999999999998e-05 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.298176+0000", 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6235999999999997e-05 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.298192+0000", 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6387000000000001e-05 2024-09-13T09:17:39.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.298319+0000", 2024-09-13T09:17:39.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012676399999999999 2024-09-13T09:17:39.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.298551+0000", 2024-09-13T09:17:39.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000231575 2024-09-13T09:17:39.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.298891+0000", 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033993199999999998 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.300228+0000", 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001337261 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.300885+0000", 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00065732099999999999 2024-09-13T09:17:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.300998+0000", 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113007 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.301085+0000", 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6643000000000004e-05 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.855 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:2933 3.19 3:985ce3d7:::rbd_data.118591b29fad.000000000000008f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 114688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:39.855 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:38.321732+0000", 2024-09-13T09:17:39.855 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.4230151740000001, 2024-09-13T09:17:39.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0032312859999999999, 2024-09-13T09:17:39.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.855 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.855 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2933 2024-09-13T09:17:39.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.856 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.321732+0000", 2024-09-13T09:17:39.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.321732+0000", 2024-09-13T09:17:39.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.321730+0000", 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.321743+0000", 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2853e-05 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.321774+0000", 2024-09-13T09:17:39.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0830000000000001e-05 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.321795+0000", 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1066000000000001e-05 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.321898+0000", 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103304 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.322090+0000", 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019191499999999999 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.322381+0000", 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029063500000000003 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.323728+0000", 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00134735 2024-09-13T09:17:39.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.324810+0000", 2024-09-13T09:17:39.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001081703 2024-09-13T09:17:39.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.324902+0000", 2024-09-13T09:17:39.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2492999999999997e-05 2024-09-13T09:17:39.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.324964+0000", 2024-09-13T09:17:39.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000006e-05 2024-09-13T09:17:39.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.861 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.861 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3155 3.6 3:65213a13:::rbd_data.118591b29fad.00000000000002cf:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:39.861 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:38.711562+0000", 2024-09-13T09:17:39.861 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.033185759, 2024-09-13T09:17:39.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0052538059999999998, 2024-09-13T09:17:39.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3155 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.711562+0000", 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.711562+0000", 2024-09-13T09:17:39.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.711557+0000", 2024-09-13T09:17:39.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999952 2024-09-13T09:17:39.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.711588+0000", 2024-09-13T09:17:39.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.061e-05 2024-09-13T09:17:39.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.711638+0000", 2024-09-13T09:17:39.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0652999999999998e-05 2024-09-13T09:17:39.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.711665+0000", 2024-09-13T09:17:39.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6686000000000002e-05 2024-09-13T09:17:39.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.711781+0000", 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115843 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.712107+0000", 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000326657 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.712964+0000", 2024-09-13T09:17:39.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085683999999999999 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.716526+0000", 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035618519999999999 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.716731+0000", 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020456900000000001 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.716777+0000", 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5921999999999998e-05 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.716815+0000", 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8804000000000003e-05 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3179 3.4 3:2748ed7f:::rbd_data.118591b29fad.0000000000000215:head [set-alloc-hint object_size 262144 write_size 262144,write 201409~3391] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:39.867 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:38.758931+0000", 2024-09-13T09:17:39.868 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.98581591000000002, 2024-09-13T09:17:39.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033936890000000001, 2024-09-13T09:17:39.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:39.868 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:39.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:39.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:39.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:39.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3179 2024-09-13T09:17:39.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.868 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:39.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.758931+0000", 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.758931+0000", 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.758929+0000", 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.758944+0000", 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4859e-05 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.758973+0000", 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9646999999999999e-05 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.758986+0000", 2024-09-13T09:17:39.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2649e-05 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.759064+0000", 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7824999999999994e-05 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.759224+0000", 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016054799999999999 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.759362+0000", 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013756400000000001 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.760925+0000", 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015631530000000001 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.762233+0000", 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013078390000000001 2024-09-13T09:17:39.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.762284+0000", 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1450999999999999e-05 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.762325+0000", 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0821000000000001e-05 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:39.874 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:39.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:17:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:marked in osd.0. 2024-09-13T09:17:39.876 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-13T09:17:39.877 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:39.877 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:39.877 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:39.877 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:39.878 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:39.870+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:39.886 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:39.886 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:39.904 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:39.904 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:39.924 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:39.925 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:39.945 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:39.945 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:39.965 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:39.966 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:39.976 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:39.970+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:39.985 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:39.986 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:40.004 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:40.005 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:40.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:17:40.110 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:40.108+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:40.178 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:40.176+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:40.188 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:40.188 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:40.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:17:40.278 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:40.274+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:40.374 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:40.374 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:40.379 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:40.378+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:40.388 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:17:40.479 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:40.474+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:40.514 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:40.514 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:40.529 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:17:40.580 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:40.578+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:40.655 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:40.655 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:40.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:17:40.714 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:40.712+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:40.785 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:40.778+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:40.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:17:40.815 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:40.816 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:40.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:17:40.883 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:40.878+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:40.974 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:40.974 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:40.985 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:40.978+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:40.988 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:17:41.086 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:41.084+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:41.155 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:41.155 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:41.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:17:41.218 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:41.214+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:41.287 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:41.284+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:41.298 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:41.298 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:41.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:17:41.389 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:41.384+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:41.481 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:41.481 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:41.489 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:41.486+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:41.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:17:41.590 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:41.588+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:41.656 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:41.656 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:41.671 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:17:41.692 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:41.688+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-13T09:17:41.793 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:41.786+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:41.800 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:41.800 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:41.815 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:17:41.894 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:41.892+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:41.988 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:41.988 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:41.995 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:41.992+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:42.003 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:17:42.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:17:42.096 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:42.092+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:42.129 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:42.129 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:42.144 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:17:42.162 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:42.178 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:42.194 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:42.196 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:42.190+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:42.211 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:42.230 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:42.230 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:42.251 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:42.251 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:42.272 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:42.277 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:42.277 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:17:42.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:17:42.277 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:17:42.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.277 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:1207 3.a 3:548359ed:::rbd_data.1185192b47c4.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:17:42.277 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:33.697882+0000", 2024-09-13T09:17:42.277 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.5747899679999993, 2024-09-13T09:17:42.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030789060000000002, 2024-09-13T09:17:42.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1207 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.697882+0000", 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.697882+0000", 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.697879+0000", 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.697898+0000", 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9392999999999999e-05 2024-09-13T09:17:42.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.697932+0000", 2024-09-13T09:17:42.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4017000000000003e-05 2024-09-13T09:17:42.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.697949+0000", 2024-09-13T09:17:42.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6616000000000002e-05 2024-09-13T09:17:42.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.698043+0000", 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4673000000000006e-05 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.698282+0000", 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000238371 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.698553+0000", 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027166100000000003 2024-09-13T09:17:42.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700302+0000", 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017481860000000001 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700783+0000", 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00048154699999999998 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700864+0000", 2024-09-13T09:17:42.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0778999999999993e-05 2024-09-13T09:17:42.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.700960+0000", 2024-09-13T09:17:42.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6545999999999993e-05 2024-09-13T09:17:42.283 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.283 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.283 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.283 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:1210 3.a 3:548359ed:::rbd_data.1185192b47c4.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:33.729887+0000", 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.5427843320000001, 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051157470000000003, 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1210 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.729887+0000", 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.729887+0000", 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.729884+0000", 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.729901+0000", 2024-09-13T09:17:42.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7373000000000001e-05 2024-09-13T09:17:42.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.729945+0000", 2024-09-13T09:17:42.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3226000000000003e-05 2024-09-13T09:17:42.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.729968+0000", 2024-09-13T09:17:42.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3822999999999999e-05 2024-09-13T09:17:42.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.730081+0000", 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113066 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.730093+0000", 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1727e-05 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.732734+0000", 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026410800000000001 2024-09-13T09:17:42.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.732816+0000", 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1291999999999993e-05 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.732918+0000", 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102373 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.734007+0000", 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001089338 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.734873+0000", 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00086618600000000002 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.734951+0000", 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7440000000000004e-05 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:33.735003+0000", 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2055999999999998e-05 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:1522 3.a 3:548359ed:::rbd_data.1185192b47c4.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1994137~18216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:34.524257+0000", 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.7484141189999995, 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031142420000000001, 2024-09-13T09:17:42.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1522 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524257+0000", 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524257+0000", 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524255+0000", 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524305+0000", 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9509999999999999e-05 2024-09-13T09:17:42.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524338+0000", 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3596999999999997e-05 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524353+0000", 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5296000000000001e-05 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524488+0000", 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013414199999999999 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.524725+0000", 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023750900000000001 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.525136+0000", 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041086300000000003 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.526705+0000", 2024-09-13T09:17:42.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015691310000000001 2024-09-13T09:17:42.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527242+0000", 2024-09-13T09:17:42.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00053680199999999996 2024-09-13T09:17:42.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527318+0000", 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6156999999999996e-05 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.527372+0000", 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3562e-05 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:1525 3.a 3:548359ed:::rbd_data.1185192b47c4.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:34.544428+0000", 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.7282433890000002, 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0033108690000000001, 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1525 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544428+0000", 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544428+0000", 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544426+0000", 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:42.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544438+0000", 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2717e-05 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544471+0000", 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2592999999999998e-05 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544505+0000", 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4409999999999998e-05 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544630+0000", 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124769 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.544853+0000", 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000223255 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.545134+0000", 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028065499999999999 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.546554+0000", 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014199390000000001 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547592+0000", 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010378049999999999 2024-09-13T09:17:42.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547672+0000", 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9806000000000001e-05 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:34.547739+0000", 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7290000000000001e-05 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:2440 3.10 3:0fee2e6e:::rbd_data.118591b29fad.000000000000028e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 73728~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:37.979084+0000", 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.2935877299999996, 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030802109999999998, 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2440 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.979084+0000", 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.979084+0000", 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.979080+0000", 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999962 2024-09-13T09:17:42.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.979109+0000", 2024-09-13T09:17:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9356000000000002e-05 2024-09-13T09:17:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.979149+0000", 2024-09-13T09:17:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9286999999999998e-05 2024-09-13T09:17:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.979173+0000", 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4358999999999998e-05 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.979252+0000", 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8875000000000006e-05 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.979512+0000", 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000260091 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.979848+0000", 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033611999999999997 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.981142+0000", 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001294391 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.982081+0000", 2024-09-13T09:17:42.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000939024 2024-09-13T09:17:42.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.982131+0000", 2024-09-13T09:17:42.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9376000000000003e-05 2024-09-13T09:17:42.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:37.982164+0000", 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3225000000000001e-05 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:2560 3.10 3:0942a527:::rbd_data.118591b29fad.000000000000039f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 106496~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.060329+0000", 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.2123420810000001, 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003071299, 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2560 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.060329+0000", 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.060329+0000", 2024-09-13T09:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.060328+0000", 2024-09-13T09:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.060339+0000", 2024-09-13T09:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.082e-05 2024-09-13T09:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.060356+0000", 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7104000000000001e-05 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.060363+0000", 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7600000000000002e-06 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.060467+0000", 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010306000000000001 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.060745+0000", 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027834099999999998 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.061200+0000", 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000455637 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.062225+0000", 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010248340000000001 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.063269+0000", 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010435310000000001 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.063361+0000", 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1664999999999997e-05 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.063401+0000", 2024-09-13T09:17:42.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0197999999999997e-05 2024-09-13T09:17:42.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.316 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.316 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:2671 3.10 3:0fee2e6e:::rbd_data.118591b29fad.000000000000028e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:42.316 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.106976+0000", 2024-09-13T09:17:42.316 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.165695618, 2024-09-13T09:17:42.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031425749999999999, 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2671 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.106976+0000", 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.106976+0000", 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.106974+0000", 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.106984+0000", 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8800000000000003e-06 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.107003+0000", 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9553999999999999e-05 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.107015+0000", 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1443e-05 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.107078+0000", 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3862000000000006e-05 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.107154+0000", 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5290999999999995e-05 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.107288+0000", 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134012 2024-09-13T09:17:42.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.109076+0000", 2024-09-13T09:17:42.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001787779 2024-09-13T09:17:42.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.110010+0000", 2024-09-13T09:17:42.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00093463499999999996 2024-09-13T09:17:42.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.110073+0000", 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3048999999999999e-05 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.110118+0000", 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5235999999999999e-05 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.322 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:2777 3.3 e31/22 3:c58f0f32:::rbd_data.118591b29fad.000000000000012b:head v 31'20, mlcod=31'18)", 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.191892+0000", 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0807795630000001, 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003392352, 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.191892+0000", 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.191892+0000", 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.191890+0000", 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.191905+0000", 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5325999999999999e-05 2024-09-13T09:17:42.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.191936+0000", 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0422000000000001e-05 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.191950+0000", 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3953e-05 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.192119+0000", 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016888299999999999 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.192157+0000", 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8024000000000001e-05 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.195216+0000", 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030588899999999999 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.195284+0000", 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8644000000000002e-05 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:2790 3.a 3:549458f7:::rbd_data.118591b29fad.0000000000000333:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.195338+0000", 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0773331380000002, 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0032531499999999998, 2024-09-13T09:17:42.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:42.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:42.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2790 2024-09-13T09:17:42.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.328 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.195338+0000", 2024-09-13T09:17:42.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.195338+0000", 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.195336+0000", 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.195347+0000", 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0329999999999999e-05 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.195372+0000", 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5361999999999999e-05 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.195384+0000", 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1856e-05 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.195452+0000", 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7941999999999994e-05 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.195542+0000", 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0179999999999994e-05 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.195683+0000", 2024-09-13T09:17:42.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000140576 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.196788+0000", 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001105909 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.198497+0000", 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017081469999999999 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.198543+0000", 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6014999999999999e-05 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.198592+0000", 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8885999999999999e-05 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.333 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:3092 3.10 3:0bfea6c3:::rbd_data.118591b29fad.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 129130~1942] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.562323+0000", 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.7103484299999998, 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055887530000000001, 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 3092 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.562323+0000", 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.562323+0000", 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.562321+0000", 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:42.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.562331+0000", 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0118999999999999e-05 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.562349+0000", 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7541999999999999e-05 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.562366+0000", 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7448e-05 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.562481+0000", 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114481 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.562868+0000", 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038711399999999998 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.563455+0000", 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00058663900000000002 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.564737+0000", 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00128268 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.567820+0000", 2024-09-13T09:17:42.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030830789999999999 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.567873+0000", 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2602999999999998e-05 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.567912+0000", 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8871000000000001e-05 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:3143 3.1e e31/22 3:78e6a0e6:::rbd_object_map.118591b29fad:head v 31'214, mlcod=31'210)", 2024-09-13T09:17:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.708437+0000", 2024-09-13T09:17:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.564234506, 2024-09-13T09:17:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087141300000000005, 2024-09-13T09:17:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.708437+0000", 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.708437+0000", 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.708435+0000", 2024-09-13T09:17:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.708453+0000", 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8352999999999999e-05 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.708487+0000", 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4464000000000001e-05 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.708506+0000", 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8983000000000001e-05 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.708688+0000", 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000181538 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.708746+0000", 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8383e-05 2024-09-13T09:17:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717073+0000", 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008326979 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717151+0000", 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7873000000000005e-05 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:3144 3.1e e31/22 3:78e6a0e6:::rbd_object_map.118591b29fad:head v 31'215, mlcod=31'211)", 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.710382+0000", 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5622897710000001, 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.007829697, 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.710382+0000", 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.710382+0000", 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.710380+0000", 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.710401+0000", 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1160000000000001e-05 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.710439+0000", 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7802999999999997e-05 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.710454+0000", 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5136e-05 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.710582+0000", 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012841299999999999 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.710651+0000", 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8266e-05 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.718142+0000", 2024-09-13T09:17:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0074908359999999998 2024-09-13T09:17:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.718211+0000", 2024-09-13T09:17:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9826000000000005e-05 2024-09-13T09:17:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:3145 3.1e e31/22 3:78e6a0e6:::rbd_object_map.118591b29fad:head v 31'216, mlcod=31'211)", 2024-09-13T09:17:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.711289+0000", 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.561382794, 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069721899999999996, 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.711289+0000", 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.711289+0000", 2024-09-13T09:17:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.711287+0000", 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.711301+0000", 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3147e-05 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.711325+0000", 2024-09-13T09:17:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4686999999999999e-05 2024-09-13T09:17:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.711340+0000", 2024-09-13T09:17:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4506e-05 2024-09-13T09:17:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.711521+0000", 2024-09-13T09:17:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018142500000000001 2024-09-13T09:17:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.711554+0000", 2024-09-13T09:17:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2453000000000003e-05 2024-09-13T09:17:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.718215+0000", 2024-09-13T09:17:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0066615590000000001 2024-09-13T09:17:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.718261+0000", 2024-09-13T09:17:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5717000000000003e-05 2024-09-13T09:17:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:3146 3.1e e31/22 3:78e6a0e6:::rbd_object_map.118591b29fad:head v 31'217, mlcod=31'212)", 2024-09-13T09:17:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.712738+0000", 2024-09-13T09:17:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5599336890000002, 2024-09-13T09:17:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055746040000000004, 2024-09-13T09:17:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.712738+0000", 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.712738+0000", 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.712736+0000", 2024-09-13T09:17:42.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:42.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.712752+0000", 2024-09-13T09:17:42.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5679999999999999e-05 2024-09-13T09:17:42.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.712783+0000", 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0759000000000001e-05 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.712799+0000", 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6650000000000002e-05 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.712899+0000", 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9352999999999996e-05 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.712938+0000", 2024-09-13T09:17:42.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9208999999999997e-05 2024-09-13T09:17:42.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.718262+0000", 2024-09-13T09:17:42.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053246200000000004 2024-09-13T09:17:42.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.718312+0000", 2024-09-13T09:17:42.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9945999999999999e-05 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:3147 3.1e e31/22 3:78e6a0e6:::rbd_object_map.118591b29fad:head v 31'218, mlcod=31'212)", 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.713686+0000", 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5589851599999998, 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047055910000000003, 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.713686+0000", 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.713686+0000", 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.713685+0000", 2024-09-13T09:17:42.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:42.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.713697+0000", 2024-09-13T09:17:42.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1885999999999999e-05 2024-09-13T09:17:42.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.713721+0000", 2024-09-13T09:17:42.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4159999999999999e-05 2024-09-13T09:17:42.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.713733+0000", 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1744e-05 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.713818+0000", 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5174999999999997e-05 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.713887+0000", 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9109999999999994e-05 2024-09-13T09:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.718315+0000", 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044277459999999998 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.718392+0000", 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7262999999999995e-05 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:3149 3.1e e31/22 3:78e6a0e6:::rbd_object_map.118591b29fad:head v 31'220, mlcod=31'212)", 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.717165+0000", 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5555069059999997, 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085441070000000004, 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717165+0000", 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717165+0000", 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717163+0000", 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717175+0000", 2024-09-13T09:17:42.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.182e-05 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717199+0000", 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4363999999999999e-05 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717212+0000", 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2223e-05 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717287+0000", 2024-09-13T09:17:42.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5978999999999999e-05 2024-09-13T09:17:42.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717319+0000", 2024-09-13T09:17:42.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1900000000000003e-05 2024-09-13T09:17:42.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.725634+0000", 2024-09-13T09:17:42.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083150949999999998 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.725709+0000", 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4213000000000002e-05 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:3157 3.10 3:08385cba:::rbd_data.118591b29fad.00000000000002d1:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.717710+0000", 2024-09-13T09:17:42.368 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5549614730000001, 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013229777999999999, 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 3157 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717710+0000", 2024-09-13T09:17:42.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717710+0000", 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717708+0000", 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717738+0000", 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0899999999999999e-05 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717767+0000", 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8359000000000001e-05 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717781+0000", 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.385e-05 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.717924+0000", 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014344600000000001 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.718212+0000", 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028759400000000001 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.719112+0000", 2024-09-13T09:17:42.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00090011699999999997 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.725443+0000", 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063311039999999997 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.730750+0000", 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053066989999999998 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.730869+0000", 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011910599999999999 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.730940+0000", 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1030000000000003e-05 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.374 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:3162 3.1e e31/22 3:7dc98835:::rbd_data.118591b29fad.00000000000002d6:head v 31'221, mlcod=31'218)", 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.725926+0000", 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5467459909999999, 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047548379999999999, 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.725926+0000", 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.725926+0000", 2024-09-13T09:17:42.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.725923+0000", 2024-09-13T09:17:42.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:42.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.725951+0000", 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8042999999999999e-05 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.725995+0000", 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3452999999999999e-05 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.726014+0000", 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9599e-05 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.726124+0000", 2024-09-13T09:17:42.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010995 2024-09-13T09:17:42.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.726184+0000", 2024-09-13T09:17:42.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9756000000000002e-05 2024-09-13T09:17:42.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.730595+0000", 2024-09-13T09:17:42.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044111159999999996 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.730680+0000", 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5216999999999994e-05 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:3163 3.10 3:0cc1ee9a:::rbd_data.118591b29fad.00000000000002d7:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.727139+0000", 2024-09-13T09:17:42.379 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5455323160000001, 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0043490789999999996, 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 3163 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.727139+0000", 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.727139+0000", 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.727136+0000", 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.727155+0000", 2024-09-13T09:17:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9130000000000001e-05 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.727190+0000", 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4566000000000001e-05 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.727210+0000", 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0435999999999999e-05 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.727353+0000", 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000143036 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.727705+0000", 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035152399999999998 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.728300+0000", 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000594942 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.731050+0000", 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027503330000000002 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.731366+0000", 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00031558699999999999 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.731436+0000", 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0809999999999995e-05 2024-09-13T09:17:42.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.731488+0000", 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1822e-05 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:3178 3.1e e31/22 3:78e6a0e6:::rbd_object_map.118591b29fad:head v 31'228, mlcod=31'223)", 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:38.758804+0000", 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.513867367, 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039562419999999996, 2024-09-13T09:17:42.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.386 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.758804+0000", 2024-09-13T09:17:42.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.758804+0000", 2024-09-13T09:17:42.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.758803+0000", 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.758825+0000", 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.16e-05 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.758845+0000", 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9993e-05 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.758859+0000", 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3679999999999999e-05 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.759773+0000", 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00091399600000000004 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.759829+0000", 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6607000000000002e-05 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.762680+0000", 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028509339999999998 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:38.762760+0000", 2024-09-13T09:17:42.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0239000000000002e-05 2024-09-13T09:17:42.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.390 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.390 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.390 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:42.391 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:42.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:17:42.393 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:42.386+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:42.493 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:42.486+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:42.515 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone3 from 0xbbf16a2 (overlap 0x24818ed) to 0x198de3f 2024-09-13T09:17:42.587 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:42.587 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:42.587 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:17:42.587 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:42.596 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:42.590+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:42.604 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:17:42.695 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:42.692+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:42.772 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:42.772 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:42.772 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:17:42.772 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:17:42.772 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:42.774 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-13T09:17:42.789 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:17:42.796 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:42.792+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:42.828 INFO:teuthology.orchestra.run.smithi032.stdout:813 write 0x1259523 thru 0x125a954 (0x1432 bytes) 2024-09-13T09:17:42.859 INFO:teuthology.orchestra.run.smithi032.stdout:816 writesame 0x28288ea thru 0x282dac1 data_size 0x1b(0x51d8 bytes) 2024-09-13T09:17:42.897 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:42.890+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:42.919 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:42.919 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:17:42.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:17:42.920 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:17:42.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.920 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:42.920 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:42.920 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.522365634, 2024-09-13T09:17:42.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077478195999999999, 2024-09-13T09:17:42.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.920 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.920 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:42.920 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392636+0000", 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392651+0000", 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392674+0000", 2024-09-13T09:17:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.264e-05 2024-09-13T09:17:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392691+0000", 2024-09-13T09:17:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7203000000000001e-05 2024-09-13T09:17:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392780+0000", 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8671999999999994e-05 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392794+0000", 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.466664+0000", 2024-09-13T09:17:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073869913999999995 2024-09-13T09:17:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467139+0000", 2024-09-13T09:17:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047488599999999999 2024-09-13T09:17:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467664+0000", 2024-09-13T09:17:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052530600000000002 2024-09-13T09:17:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469125+0000", 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001460432 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469847+0000", 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072197799999999996 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470006+0000", 2024-09-13T09:17:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015955900000000001 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470116+0000", 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109836 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531091+0000", 2024-09-13T09:17:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.383912662, 2024-09-13T09:17:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010986811000000001, 2024-09-13T09:17:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 34 2024-09-13T09:17:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531091+0000", 2024-09-13T09:17:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531091+0000", 2024-09-13T09:17:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531090+0000", 2024-09-13T09:17:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531096+0000", 2024-09-13T09:17:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.806e-06 2024-09-13T09:17:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531108+0000", 2024-09-13T09:17:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.224e-05 2024-09-13T09:17:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531117+0000", 2024-09-13T09:17:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6030000000000005e-06 2024-09-13T09:17:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.536352+0000", 2024-09-13T09:17:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052353260000000002 2024-09-13T09:17:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.536628+0000", 2024-09-13T09:17:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027556500000000002 2024-09-13T09:17:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:42.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.536981+0000", 2024-09-13T09:17:42.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035341700000000002 2024-09-13T09:17:42.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.541700+0000", 2024-09-13T09:17:42.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047184100000000001 2024-09-13T09:17:42.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.541795+0000", 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5031999999999994e-05 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.541955+0000", 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160056 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542078+0000", 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012296200000000001 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.935 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.936 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:42.936 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531127+0000", 2024-09-13T09:17:42.936 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.383876736000001, 2024-09-13T09:17:42.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011389865000000001, 2024-09-13T09:17:42.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:42.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 35 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531127+0000", 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531127+0000", 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531126+0000", 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531274+0000", 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014800800000000001 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531288+0000", 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3867e-05 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531297+0000", 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9330000000000007e-06 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.538727+0000", 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.007429639 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.538949+0000", 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022249499999999999 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.539160+0000", 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000211079 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542252+0000", 2024-09-13T09:17:42.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030918460000000001 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542280+0000", 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7433e-05 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542409+0000", 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012945599999999999 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542517+0000", 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107725 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531307+0000", 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.383696260000001, 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012285895, 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.942 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.943 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.943 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:42.943 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:42.943 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 36 2024-09-13T09:17:42.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.943 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531307+0000", 2024-09-13T09:17:42.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531307+0000", 2024-09-13T09:17:42.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531306+0000", 2024-09-13T09:17:42.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:42.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531313+0000", 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3130000000000004e-06 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531322+0000", 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.85e-06 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531329+0000", 2024-09-13T09:17:42.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9600000000000003e-06 2024-09-13T09:17:42.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.540210+0000", 2024-09-13T09:17:42.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088810439999999994 2024-09-13T09:17:42.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.540477+0000", 2024-09-13T09:17:42.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000267438 2024-09-13T09:17:42.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:42.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.540862+0000", 2024-09-13T09:17:42.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038524400000000003 2024-09-13T09:17:42.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543383+0000", 2024-09-13T09:17:42.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002520764 2024-09-13T09:17:42.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543447+0000", 2024-09-13T09:17:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4063e-05 2024-09-13T09:17:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543538+0000", 2024-09-13T09:17:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1268999999999996e-05 2024-09-13T09:17:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543593+0000", 2024-09-13T09:17:42.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000002e-05 2024-09-13T09:17:42.949 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.949 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.949 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.949 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:42.949 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.531337+0000", 2024-09-13T09:17:42.949 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.38366675, 2024-09-13T09:17:42.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013657717, 2024-09-13T09:17:42.949 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.949 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531337+0000", 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531337+0000", 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531336+0000", 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531344+0000", 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5599999999999996e-06 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531353+0000", 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8869999999999998e-06 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.531360+0000", 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9100000000000005e-06 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542600+0000", 2024-09-13T09:17:42.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011239638999999999 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542747+0000", 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014643499999999999 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.542947+0000", 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000200149 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.543918+0000", 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00097078599999999996 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.544760+0000", 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00084245000000000001 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.544898+0000", 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013783900000000001 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.544995+0000", 2024-09-13T09:17:42.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6766000000000002e-05 2024-09-13T09:17:42.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.955 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.955 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:42.956 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.631401+0000", 2024-09-13T09:17:42.956 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.283602253000002, 2024-09-13T09:17:42.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010376106, 2024-09-13T09:17:42.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:42.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:42.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 86 2024-09-13T09:17:42.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.956 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631401+0000", 2024-09-13T09:17:42.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631401+0000", 2024-09-13T09:17:42.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631400+0000", 2024-09-13T09:17:42.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:42.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631411+0000", 2024-09-13T09:17:42.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1029999999999999e-05 2024-09-13T09:17:42.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631429+0000", 2024-09-13T09:17:42.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7907e-05 2024-09-13T09:17:42.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631443+0000", 2024-09-13T09:17:42.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4283e-05 2024-09-13T09:17:42.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.636777+0000", 2024-09-13T09:17:42.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053336190000000004 2024-09-13T09:17:42.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.636951+0000", 2024-09-13T09:17:42.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174325 2024-09-13T09:17:42.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:42.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.637168+0000", 2024-09-13T09:17:42.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021682200000000001 2024-09-13T09:17:42.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641519+0000", 2024-09-13T09:17:42.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0043510759999999997 2024-09-13T09:17:42.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641638+0000", 2024-09-13T09:17:42.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118573 2024-09-13T09:17:42.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641721+0000", 2024-09-13T09:17:42.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2803e-05 2024-09-13T09:17:42.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641777+0000", 2024-09-13T09:17:42.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6771999999999998e-05 2024-09-13T09:17:42.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.962 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.963 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:42.963 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.631461+0000", 2024-09-13T09:17:42.963 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.283542780000001, 2024-09-13T09:17:42.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010750297000000001, 2024-09-13T09:17:42.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:42.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:42.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 87 2024-09-13T09:17:42.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.964 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631461+0000", 2024-09-13T09:17:42.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631461+0000", 2024-09-13T09:17:42.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631460+0000", 2024-09-13T09:17:42.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:42.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631618+0000", 2024-09-13T09:17:42.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015849899999999999 2024-09-13T09:17:42.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631641+0000", 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2912999999999999e-05 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631657+0000", 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6305999999999999e-05 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.638673+0000", 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0070156300000000001 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.638802+0000", 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129049 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.639028+0000", 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022574100000000001 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641921+0000", 2024-09-13T09:17:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002893648 2024-09-13T09:17:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.641947+0000", 2024-09-13T09:17:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5307e-05 2024-09-13T09:17:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642120+0000", 2024-09-13T09:17:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017344499999999999 2024-09-13T09:17:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642211+0000", 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0876000000000002e-05 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.631676+0000", 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.283327345, 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011331733, 2024-09-13T09:17:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 88 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631676+0000", 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631676+0000", 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631675+0000", 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631687+0000", 2024-09-13T09:17:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2163e-05 2024-09-13T09:17:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631705+0000", 2024-09-13T09:17:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7876999999999998e-05 2024-09-13T09:17:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.631722+0000", 2024-09-13T09:17:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.685e-05 2024-09-13T09:17:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.640114+0000", 2024-09-13T09:17:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083919979999999995 2024-09-13T09:17:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.640389+0000", 2024-09-13T09:17:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027471800000000002 2024-09-13T09:17:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.640788+0000", 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00039957000000000002 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642474+0000", 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001686014 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642883+0000", 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040869700000000002 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.642957+0000", 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4239000000000005e-05 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.643008+0000", 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0917000000000001e-05 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.797962+0000", 2024-09-13T09:17:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.117041371999999, 2024-09-13T09:17:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01079161, 2024-09-13T09:17:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 110 2024-09-13T09:17:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797962+0000", 2024-09-13T09:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797962+0000", 2024-09-13T09:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797960+0000", 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797974+0000", 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4545999999999999e-05 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.797999+0000", 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4573e-05 2024-09-13T09:17:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798017+0000", 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7940000000000001e-05 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.804966+0000", 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069496469999999998 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.805091+0000", 2024-09-13T09:17:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012414199999999999 2024-09-13T09:17:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.805276+0000", 2024-09-13T09:17:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018536799999999999 2024-09-13T09:17:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808455+0000", 2024-09-13T09:17:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00317914 2024-09-13T09:17:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808580+0000", 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012457500000000001 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808654+0000", 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4373000000000001e-05 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808754+0000", 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9869000000000002e-05 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798038+0000", 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.116965535999999, 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012627325999999999, 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 111 2024-09-13T09:17:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798038+0000", 2024-09-13T09:17:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798038+0000", 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798037+0000", 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798048+0000", 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1194e-05 2024-09-13T09:17:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798074+0000", 2024-09-13T09:17:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5516000000000001e-05 2024-09-13T09:17:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798088+0000", 2024-09-13T09:17:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4663e-05 2024-09-13T09:17:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.806212+0000", 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081233810000000007 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.806328+0000", 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011605200000000001 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.806514+0000", 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018678299999999999 2024-09-13T09:17:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808565+0000", 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020504439999999998 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810494+0000", 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019292109999999999 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810569+0000", 2024-09-13T09:17:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4436000000000004e-05 2024-09-13T09:17:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810665+0000", 2024-09-13T09:17:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6842999999999994e-05 2024-09-13T09:17:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798108+0000", 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.11689591, 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014039507999999999, 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798108+0000", 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798108+0000", 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798107+0000", 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798267+0000", 2024-09-13T09:17:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160116 2024-09-13T09:17:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798294+0000", 2024-09-13T09:17:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6923e-05 2024-09-13T09:17:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798317+0000", 2024-09-13T09:17:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3693000000000002e-05 2024-09-13T09:17:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.807303+0000", 2024-09-13T09:17:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089853680000000005 2024-09-13T09:17:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.807428+0000", 2024-09-13T09:17:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012510200000000001 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.807616+0000", 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188145 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810479+0000", 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028628949999999998 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811969+0000", 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001489661 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812047+0000", 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8296999999999999e-05 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812147+0000", 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100375 2024-09-13T09:17:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798339+0000", 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.116664312000001, 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013996181999999999, 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113 2024-09-13T09:17:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798339+0000", 2024-09-13T09:17:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798339+0000", 2024-09-13T09:17:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798338+0000", 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798357+0000", 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.897e-05 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798404+0000", 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7119000000000001e-05 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798421+0000", 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7187e-05 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808797+0000", 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010375880000000001 2024-09-13T09:17:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.808915+0000", 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011846200000000001 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809099+0000", 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183313 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811848+0000", 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027493439999999999 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812174+0000", 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032562200000000002 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812244+0000", 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0902000000000007e-05 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812335+0000", 2024-09-13T09:17:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0995999999999994e-05 2024-09-13T09:17:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798442+0000", 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.116561552, 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015723348000000002, 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798442+0000", 2024-09-13T09:17:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798442+0000", 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798441+0000", 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798453+0000", 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2784e-05 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798476+0000", 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2875999999999999e-05 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798490+0000", 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3482999999999999e-05 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809646+0000", 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011155847 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809754+0000", 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108256 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.809965+0000", 2024-09-13T09:17:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000210659 2024-09-13T09:17:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811953+0000", 2024-09-13T09:17:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019889439999999999 2024-09-13T09:17:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813962+0000", 2024-09-13T09:17:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020080850000000002 2024-09-13T09:17:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814040+0000", 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8184999999999998e-05 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814165+0000", 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012563299999999999 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798506+0000", 2024-09-13T09:17:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.116497183, 2024-09-13T09:17:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015834198000000001, 2024-09-13T09:17:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 115 2024-09-13T09:17:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798506+0000", 2024-09-13T09:17:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798506+0000", 2024-09-13T09:17:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798505+0000", 2024-09-13T09:17:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798658+0000", 2024-09-13T09:17:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000153005 2024-09-13T09:17:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798681+0000", 2024-09-13T09:17:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2634e-05 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798697+0000", 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5696e-05 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810746+0000", 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012049331 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.810873+0000", 2024-09-13T09:17:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012754899999999999 2024-09-13T09:17:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.811051+0000", 2024-09-13T09:17:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000177252 2024-09-13T09:17:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813948+0000", 2024-09-13T09:17:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028975239999999998 2024-09-13T09:17:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814188+0000", 2024-09-13T09:17:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000239724 2024-09-13T09:17:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814255+0000", 2024-09-13T09:17:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7372999999999993e-05 2024-09-13T09:17:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814341+0000", 2024-09-13T09:17:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5270000000000002e-05 2024-09-13T09:17:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798716+0000", 2024-09-13T09:17:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.116287337999999, 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015827386999999998, 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798716+0000", 2024-09-13T09:17:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798716+0000", 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798715+0000", 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798727+0000", 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1800000000000001e-05 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798746+0000", 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8992999999999999e-05 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798765+0000", 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9049e-05 2024-09-13T09:17:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812383+0000", 2024-09-13T09:17:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013618531999999999 2024-09-13T09:17:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812498+0000", 2024-09-13T09:17:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115056 2024-09-13T09:17:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.812682+0000", 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018366899999999999 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814367+0000", 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00168501 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814421+0000", 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4199000000000001e-05 2024-09-13T09:17:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814492+0000", 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0502999999999999e-05 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814544+0000", 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2039000000000001e-05 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.798781+0000", 2024-09-13T09:17:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.116222451999999, 2024-09-13T09:17:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016163997999999999, 2024-09-13T09:17:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 117 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798781+0000", 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798781+0000", 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798780+0000", 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798795+0000", 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4574e-05 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798813+0000", 2024-09-13T09:17:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8536e-05 2024-09-13T09:17:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.798829+0000", 2024-09-13T09:17:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5866999999999999e-05 2024-09-13T09:17:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813236+0000", 2024-09-13T09:17:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014407229000000001 2024-09-13T09:17:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813350+0000", 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011376500000000001 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.813530+0000", 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179619 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814654+0000", 2024-09-13T09:17:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011244339999999999 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814768+0000", 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011397599999999999 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814845+0000", 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7299999999999995e-05 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.814945+0000", 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9875000000000002e-05 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:12.288452+0000", 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.626551766999999, 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010418200000000001, 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140 2024-09-13T09:17:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288452+0000", 2024-09-13T09:17:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288452+0000", 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288451+0000", 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288508+0000", 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6706999999999998e-05 2024-09-13T09:17:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288517+0000", 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8339999999999994e-06 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288523+0000", 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5419999999999996e-06 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.291720+0000", 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031973499999999998 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.291765+0000", 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.443e-05 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.291834+0000", 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9053000000000004e-05 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.293879+0000", 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020453619999999998 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.298804+0000", 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049248069999999998 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.298831+0000", 2024-09-13T09:17:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7436e-05 2024-09-13T09:17:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.298870+0000", 2024-09-13T09:17:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8549e-05 2024-09-13T09:17:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:12.288530+0000", 2024-09-13T09:17:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.62647381, 2024-09-13T09:17:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011049003999999999, 2024-09-13T09:17:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 141 2024-09-13T09:17:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288530+0000", 2024-09-13T09:17:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288530+0000", 2024-09-13T09:17:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288529+0000", 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288532+0000", 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288536+0000", 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8480000000000003e-06 2024-09-13T09:17:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.288539+0000", 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0479999999999999e-06 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.292188+0000", 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036491639999999999 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.292357+0000", 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016818200000000001 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.292492+0000", 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013562099999999999 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.293885+0000", 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001393044 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.299519+0000", 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056339670000000001 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.299547+0000", 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7928000000000001e-05 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:12.299579+0000", 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1547999999999997e-05 2024-09-13T09:17:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.874903452000002, 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010963133, 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 158 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040100+0000", 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040255+0000", 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015546599999999999 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040268+0000", 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2860000000000001e-05 2024-09-13T09:17:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040276+0000", 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5399999999999996e-06 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.047322+0000", 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0070454380000000002 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:43.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.047443+0000", 2024-09-13T09:17:43.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012126 2024-09-13T09:17:43.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:43.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.047631+0000", 2024-09-13T09:17:43.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000187558 2024-09-13T09:17:43.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:43.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050775+0000", 2024-09-13T09:17:43.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031440000000000001 2024-09-13T09:17:43.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050797+0000", 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2730000000000001e-05 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050928+0000", 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013016500000000001 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:43.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051063+0000", 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013571300000000001 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:13.040286+0000", 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.87471729, 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011356279, 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.046 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 159 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040286+0000", 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:43.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040286+0000", 2024-09-13T09:17:43.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:43.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:43.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040285+0000", 2024-09-13T09:17:43.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:43.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:43.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040296+0000", 2024-09-13T09:17:43.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0473e-05 2024-09-13T09:17:43.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040305+0000", 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0070000000000003e-06 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.040315+0000", 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8500000000000006e-06 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:43.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.049739+0000", 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094246899999999995 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.049892+0000", 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015284900000000001 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.050097+0000", 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020424500000000001 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051387+0000", 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012901970000000001 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051511+0000", 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124039 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051589+0000", 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.873e-05 2024-09-13T09:17:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:13.051643+0000", 2024-09-13T09:17:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3102000000000001e-05 2024-09-13T09:17:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:43.052 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:43.054 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:43.050+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:43.054 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:17:43.154 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:43.150+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:43.189 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3.f0f996b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.32885182699999999, 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.328876416, 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:17:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:17:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:17:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 1 2024-09-13T09:17:43.194 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:43.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:17:43.255 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:43.252+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:43.330 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:43.331 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:43.344 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:17:43.356 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:43.350+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:43.457 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:43.452+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:43.510 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:43.510 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:43.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:43.510 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:43.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.510 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:129 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'87, mlcod=16'85)", 2024-09-13T09:17:43.510 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.288944+0000", 2024-09-13T09:17:43.510 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.219081736, 2024-09-13T09:17:43.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0048963449999999999, 2024-09-13T09:17:43.510 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.510 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288944+0000", 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288944+0000", 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288942+0000", 2024-09-13T09:17:43.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:43.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288960+0000", 2024-09-13T09:17:43.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8077e-05 2024-09-13T09:17:43.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.288989+0000", 2024-09-13T09:17:43.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9476e-05 2024-09-13T09:17:43.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289003+0000", 2024-09-13T09:17:43.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3046e-05 2024-09-13T09:17:43.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289103+0000", 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010054 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289137+0000", 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3575999999999998e-05 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293750+0000", 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046137610000000001 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293840+0000", 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9698999999999996e-05 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:131 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.289777+0000", 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.218248783, 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005922531, 2024-09-13T09:17:43.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.516 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289777+0000", 2024-09-13T09:17:43.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289777+0000", 2024-09-13T09:17:43.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289775+0000", 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289961+0000", 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018587200000000001 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.289993+0000", 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2007e-05 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290016+0000", 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3025999999999999e-05 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290280+0000", 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026384800000000001 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290318+0000", 2024-09-13T09:17:43.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7945999999999999e-05 2024-09-13T09:17:43.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295642+0000", 2024-09-13T09:17:43.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0053246159999999999 2024-09-13T09:17:43.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295699+0000", 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6898999999999999e-05 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:132 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.290371+0000", 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.217654333999999, 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0053750650000000001, 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.521 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290371+0000", 2024-09-13T09:17:43.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290371+0000", 2024-09-13T09:17:43.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290369+0000", 2024-09-13T09:17:43.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:43.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290384+0000", 2024-09-13T09:17:43.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5133999999999999e-05 2024-09-13T09:17:43.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290419+0000", 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4555999999999999e-05 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290437+0000", 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8343e-05 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291376+0000", 2024-09-13T09:17:43.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00093908299999999995 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291410+0000", 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3883000000000001e-05 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295705+0000", 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042944790000000004 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295746+0000", 2024-09-13T09:17:43.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1496999999999999e-05 2024-09-13T09:17:43.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.525 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.525 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:133 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-13T09:17:43.525 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.290748+0000", 2024-09-13T09:17:43.525 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.217277313, 2024-09-13T09:17:43.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0050401270000000001, 2024-09-13T09:17:43.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290748+0000", 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290748+0000", 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290747+0000", 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290761+0000", 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4236000000000001e-05 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290788+0000", 2024-09-13T09:17:43.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6826999999999999e-05 2024-09-13T09:17:43.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.290804+0000", 2024-09-13T09:17:43.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5798999999999999e-05 2024-09-13T09:17:43.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291981+0000", 2024-09-13T09:17:43.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011771080000000001 2024-09-13T09:17:43.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292015+0000", 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4693000000000001e-05 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295750+0000", 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037345780000000001 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295788+0000", 2024-09-13T09:17:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:134 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.291161+0000", 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.216864587, 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064130539999999996, 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291161+0000", 2024-09-13T09:17:43.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291161+0000", 2024-09-13T09:17:43.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291159+0000", 2024-09-13T09:17:43.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:43.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291257+0000", 2024-09-13T09:17:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7498999999999996e-05 2024-09-13T09:17:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291284+0000", 2024-09-13T09:17:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:17:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291305+0000", 2024-09-13T09:17:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1773e-05 2024-09-13T09:17:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292753+0000", 2024-09-13T09:17:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014473719999999999 2024-09-13T09:17:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292786+0000", 2024-09-13T09:17:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3370000000000001e-05 2024-09-13T09:17:43.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297518+0000", 2024-09-13T09:17:43.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0047318569999999999 2024-09-13T09:17:43.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297574+0000", 2024-09-13T09:17:43.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6026000000000002e-05 2024-09-13T09:17:43.534 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.534 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.535 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:135 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'85)", 2024-09-13T09:17:43.535 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.291641+0000", 2024-09-13T09:17:43.535 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.216384486999999, 2024-09-13T09:17:43.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0059934710000000002, 2024-09-13T09:17:43.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.535 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.535 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291641+0000", 2024-09-13T09:17:43.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291641+0000", 2024-09-13T09:17:43.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291639+0000", 2024-09-13T09:17:43.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:43.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291653+0000", 2024-09-13T09:17:43.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3977e-05 2024-09-13T09:17:43.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291684+0000", 2024-09-13T09:17:43.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0298999999999999e-05 2024-09-13T09:17:43.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.291701+0000", 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7847e-05 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294015+0000", 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002313402 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294047+0000", 2024-09-13T09:17:43.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2063000000000001e-05 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297582+0000", 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035351430000000001 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297634+0000", 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2460000000000003e-05 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:136 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'85)", 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.292054+0000", 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.215971112999998, 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0073146829999999998, 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292054+0000", 2024-09-13T09:17:43.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292054+0000", 2024-09-13T09:17:43.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292053+0000", 2024-09-13T09:17:43.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:43.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292067+0000", 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.411e-05 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292092+0000", 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5173000000000001e-05 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292111+0000", 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8963000000000001e-05 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294704+0000", 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025933800000000002 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294735+0000", 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0366e-05 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299316+0000", 2024-09-13T09:17:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004581412 2024-09-13T09:17:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299369+0000", 2024-09-13T09:17:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3016000000000002e-05 2024-09-13T09:17:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:137 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'85)", 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.292515+0000", 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.215510903999999, 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068905700000000004, 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292515+0000", 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292515+0000", 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292513+0000", 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292586+0000", 2024-09-13T09:17:43.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3139999999999994e-05 2024-09-13T09:17:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292618+0000", 2024-09-13T09:17:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1878999999999998e-05 2024-09-13T09:17:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292639+0000", 2024-09-13T09:17:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0752999999999999e-05 2024-09-13T09:17:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295880+0000", 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0032411889999999998 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.295927+0000", 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7175999999999997e-05 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299375+0000", 2024-09-13T09:17:43.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0034476400000000001 2024-09-13T09:17:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299405+0000", 2024-09-13T09:17:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0642999999999998e-05 2024-09-13T09:17:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:138 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'85)", 2024-09-13T09:17:43.549 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.292944+0000", 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.215081269999999, 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064997630000000004, 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292944+0000", 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292944+0000", 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292943+0000", 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292957+0000", 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4610000000000001e-05 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.292982+0000", 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5003000000000001e-05 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293000+0000", 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8142999999999999e-05 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.296939+0000", 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0039383700000000001 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.296972+0000", 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2716999999999999e-05 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299412+0000", 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024409340000000001 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299444+0000", 2024-09-13T09:17:43.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.154e-05 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:139 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'85)", 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.293238+0000", 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.214787386000001, 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062557749999999999, 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293238+0000", 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293238+0000", 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293237+0000", 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293250+0000", 2024-09-13T09:17:43.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.379e-05 2024-09-13T09:17:43.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293276+0000", 2024-09-13T09:17:43.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6126e-05 2024-09-13T09:17:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293295+0000", 2024-09-13T09:17:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8904000000000001e-05 2024-09-13T09:17:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297711+0000", 2024-09-13T09:17:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044153789999999997 2024-09-13T09:17:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.297745+0000", 2024-09-13T09:17:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4499999999999998e-05 2024-09-13T09:17:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299450+0000", 2024-09-13T09:17:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00170436 2024-09-13T09:17:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299494+0000", 2024-09-13T09:17:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4302999999999999e-05 2024-09-13T09:17:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'85)", 2024-09-13T09:17:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.293606+0000", 2024-09-13T09:17:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.214419109000001, 2024-09-13T09:17:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066098429999999998, 2024-09-13T09:17:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.559 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293606+0000", 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293606+0000", 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293605+0000", 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293707+0000", 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010204600000000001 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293732+0000", 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5429999999999999e-05 2024-09-13T09:17:43.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.293749+0000", 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6390000000000001e-05 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.298325+0000", 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045769440000000003 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.298370+0000", 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4187000000000002e-05 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300157+0000", 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017874690000000001 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300216+0000", 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9067000000000002e-05 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4185.0:141 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'85)", 2024-09-13T09:17:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:12.294217+0000", 2024-09-13T09:17:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.213808874000001, 2024-09-13T09:17:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067059559999999999, 2024-09-13T09:17:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294217+0000", 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294217+0000", 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294215+0000", 2024-09-13T09:17:43.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294229+0000", 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3974e-05 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294261+0000", 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1903000000000003e-05 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.294280+0000", 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9589000000000002e-05 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299531+0000", 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0052504029999999998 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.299567+0000", 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6427000000000002e-05 2024-09-13T09:17:43.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300868+0000", 2024-09-13T09:17:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013010470000000001 2024-09-13T09:17:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:12.300923+0000", 2024-09-13T09:17:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4203000000000003e-05 2024-09-13T09:17:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3302 3.6 3:656e33c6:::rbd_data.118591b29fad.0000000000000073:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.520911+0000", 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.98711464400000004, 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0054980039999999999, 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3302 2024-09-13T09:17:43.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.520911+0000", 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.520911+0000", 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.520908+0000", 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.520922+0000", 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.332e-05 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.520962+0000", 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0256000000000002e-05 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.520982+0000", 2024-09-13T09:17:43.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0227000000000001e-05 2024-09-13T09:17:43.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.521152+0000", 2024-09-13T09:17:43.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016965599999999999 2024-09-13T09:17:43.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.521571+0000", 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00041941299999999999 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.526409+0000", 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0048376219999999998 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3319 3.6 3:656e33c6:::rbd_data.118591b29fad.0000000000000073:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:17:43.573 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.529366+0000", 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97865929600000001, 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062933620000000003, 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3319 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.529366+0000", 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.529366+0000", 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.529364+0000", 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.529380+0000", 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5829999999999999e-05 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.529411+0000", 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1164000000000002e-05 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.529425+0000", 2024-09-13T09:17:43.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4552999999999999e-05 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.529549+0000", 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000123416 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.529803+0000", 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025447399999999998 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:17:43.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.530427+0000", 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00062387499999999997 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.532077+0000", 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001649965 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535422+0000", 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033445039999999999 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535561+0000", 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000139615 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535660+0000", 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8222999999999998e-05 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3329 3.10 e35/33 3:0bfea6c3:::rbd_data.118591b29fad.0000000000000082:head v 35'22, mlcod=31'18)", 2024-09-13T09:17:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.537215+0000", 2024-09-13T09:17:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97081054600000005, 2024-09-13T09:17:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077874559999999999, 2024-09-13T09:17:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537215+0000", 2024-09-13T09:17:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537215+0000", 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537213+0000", 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537234+0000", 2024-09-13T09:17:43.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0769000000000001e-05 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537262+0000", 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7849999999999999e-05 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537280+0000", 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8627e-05 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537418+0000", 2024-09-13T09:17:43.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000138052 2024-09-13T09:17:43.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537470+0000", 2024-09-13T09:17:43.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1882999999999998e-05 2024-09-13T09:17:43.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.544790+0000", 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0073199670000000001 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.545002+0000", 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021217799999999999 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3330 3.17 e35/34 3:e9b422bc:::rbd_data.118591b29fad.0000000000000083:head v 35'110, mlcod=0'0)", 2024-09-13T09:17:43.584 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.537862+0000", 2024-09-13T09:17:43.585 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97016381500000004, 2024-09-13T09:17:43.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0052922890000000004, 2024-09-13T09:17:43.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.585 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.585 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537862+0000", 2024-09-13T09:17:43.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537862+0000", 2024-09-13T09:17:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537859+0000", 2024-09-13T09:17:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537878+0000", 2024-09-13T09:17:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8315999999999999e-05 2024-09-13T09:17:43.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537906+0000", 2024-09-13T09:17:43.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8189999999999999e-05 2024-09-13T09:17:43.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.537924+0000", 2024-09-13T09:17:43.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7533e-05 2024-09-13T09:17:43.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.538061+0000", 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013782200000000001 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.538120+0000", 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8542999999999998e-05 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.543047+0000", 2024-09-13T09:17:43.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0049267119999999998 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.543154+0000", 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107359 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3331 3.b e35/33 3:d18d3b13:::rbd_data.118591b29fad.0000000000000084:head v 35'24, mlcod=31'21)", 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.539412+0000", 2024-09-13T09:17:43.589 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.96861363700000003, 2024-09-13T09:17:43.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005520891, 2024-09-13T09:17:43.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.590 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.590 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539412+0000", 2024-09-13T09:17:43.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539412+0000", 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539409+0000", 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539428+0000", 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9023000000000001e-05 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539464+0000", 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5969000000000001e-05 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539487+0000", 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2824e-05 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539581+0000", 2024-09-13T09:17:43.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3509000000000002e-05 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539629+0000", 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8216000000000001e-05 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.544833+0000", 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0052046929999999998 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.544933+0000", 2024-09-13T09:17:43.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9302999999999994e-05 2024-09-13T09:17:43.594 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.594 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.594 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.594 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3346 3.12 3:49826968:::rbd_data.118591b29fad.000000000000008d:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:17:43.594 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.546204+0000", 2024-09-13T09:17:43.594 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.96182118299999997, 2024-09-13T09:17:43.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0048161000000000002, 2024-09-13T09:17:43.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.594 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3346 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.546204+0000", 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.546204+0000", 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.546201+0000", 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.546216+0000", 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.505e-05 2024-09-13T09:17:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.546264+0000", 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7519999999999999e-05 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.546285+0000", 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1063000000000001e-05 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.546631+0000", 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00034641000000000001 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.546839+0000", 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020806500000000001 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.547324+0000", 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00048489699999999998 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:43.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.549868+0000", 2024-09-13T09:17:43.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025438499999999998 2024-09-13T09:17:43.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:43.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.550782+0000", 2024-09-13T09:17:43.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091368299999999999 2024-09-13T09:17:43.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.550925+0000", 2024-09-13T09:17:43.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014311900000000001 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.551020+0000", 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5369000000000001e-05 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3371 3.b e35/33 3:d2fc5ee3:::rbd_data.118591b29fad.00000000000002cb:head v 35'26, mlcod=31'22)", 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.560240+0000", 2024-09-13T09:17:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.94778560599999995, 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0047605879999999996, 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560240+0000", 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560240+0000", 2024-09-13T09:17:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560239+0000", 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560287+0000", 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7482999999999999e-05 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560317+0000", 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0494e-05 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560331+0000", 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4034e-05 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560480+0000", 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014841299999999999 2024-09-13T09:17:43.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560528+0000", 2024-09-13T09:17:43.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8236999999999999e-05 2024-09-13T09:17:43.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.564921+0000", 2024-09-13T09:17:43.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0043933430000000001 2024-09-13T09:17:43.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.565000+0000", 2024-09-13T09:17:43.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9375999999999994e-05 2024-09-13T09:17:43.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.605 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.605 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3376 3.1 3:85f3995d:::rbd_data.118591b29fad.00000000000002d0:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:17:43.605 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.563523+0000", 2024-09-13T09:17:43.605 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.94450257900000001, 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0047032539999999996, 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3376 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.563523+0000", 2024-09-13T09:17:43.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.563523+0000", 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.563521+0000", 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:43.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.563534+0000", 2024-09-13T09:17:43.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3763e-05 2024-09-13T09:17:43.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:43.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.563576+0000", 2024-09-13T09:17:43.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1479e-05 2024-09-13T09:17:43.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:43.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.563601+0000", 2024-09-13T09:17:43.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5194e-05 2024-09-13T09:17:43.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:43.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.563700+0000", 2024-09-13T09:17:43.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9060000000000004e-05 2024-09-13T09:17:43.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:43.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.563896+0000", 2024-09-13T09:17:43.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196195 2024-09-13T09:17:43.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 4", 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.565016+0000", 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011195020000000001 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.567489+0000", 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002473463 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:43.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.568068+0000", 2024-09-13T09:17:43.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00057848800000000005 2024-09-13T09:17:43.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:43.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.568157+0000", 2024-09-13T09:17:43.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9087999999999998e-05 2024-09-13T09:17:43.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:43.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:43.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:43.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.568226+0000", 2024-09-13T09:17:43.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9321999999999998e-05 2024-09-13T09:17:43.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.612 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:43.612 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:43.612 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:43.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:17:43.614 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:43.608+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:43.714 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:43.712+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:43.791 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:43.791 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:43.791 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:43.791 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:43.809 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:17:43.811 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:17:43.816 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:43.810+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:43.831 INFO:teuthology.orchestra.run.smithi032.stdout:817 writesame 0xea23042 thru 0xea25b06 data_size 0x1(0x2ac5 bytes) 2024-09-13T09:17:43.842 INFO:teuthology.orchestra.run.smithi032.stdout:818 trunc from 0xea25b07 to 0x8684c34 2024-09-13T09:17:43.870 INFO:teuthology.orchestra.run.smithi032.stdout:819 write 0x8d6e662 thru 0x8d75d9f (0x773e bytes) 2024-09-13T09:17:43.876 INFO:teuthology.orchestra.run.smithi032.stdout:820 write 0x1c99417 thru 0x1ca00ad (0x6c97 bytes) 2024-09-13T09:17:43.899 INFO:teuthology.orchestra.run.smithi032.stdout:821 write 0xe4b8baf thru 0xe4c6088 (0xd4da bytes) 2024-09-13T09:17:43.905 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:43.905 INFO:teuthology.orchestra.run.smithi032.stdout:823 read 0xbc5dc92 thru 0xbc6a9a8 (0xcd17 bytes) 2024-09-13T09:17:43.905 INFO:teuthology.orchestra.run.smithi032.stdout:824 read 0x37ff394 thru 0x3805b23 (0x6790 bytes) 2024-09-13T09:17:43.905 INFO:teuthology.orchestra.run.smithi032.stdout:826 read 0x22f8c8f thru 0x22fca33 (0x3da5 bytes) 2024-09-13T09:17:43.906 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:43.906 INFO:teuthology.orchestra.run.smithi032.stdout:828 writesame 0x589e604 thru 0x58a1677 data_size 0xe(0x3074 bytes) 2024-09-13T09:17:43.913 INFO:teuthology.orchestra.run.smithi032.stdout:829 write 0x236de06 thru 0x23702cf (0x24ca bytes) 2024-09-13T09:17:43.916 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:43.912+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:43.926 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:43.953 INFO:teuthology.orchestra.run.smithi032.stdout:832 trunc from 0xe4c6089 to 0x5c2e6a 2024-09-13T09:17:43.971 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:43.971 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:43.997 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:17:44.017 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:44.010+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:44.087 INFO:teuthology.orchestra.run.smithi032.stdout:833 writesame 0x898fbcd thru 0x89955ce data_size 0x29(0x5a02 bytes) 2024-09-13T09:17:44.088 INFO:teuthology.orchestra.run.smithi032.stdout:834 writesame 0x4a76891 thru 0x4a7e97a data_size 0x1d(0x80ea bytes) 2024-09-13T09:17:44.092 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:44.092 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:44.092 INFO:teuthology.orchestra.run.smithi032.stdout:838 read 0x4b51d9b thru 0x4b5ec6f (0xced5 bytes) 2024-09-13T09:17:44.092 INFO:teuthology.orchestra.run.smithi032.stdout:839 write 0x22784f8 thru 0x2278cd7 (0x7e0 bytes) 2024-09-13T09:17:44.098 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:44.098 INFO:teuthology.orchestra.run.smithi032.stdout:841 read 0x29616a3 thru 0x2968033 (0x6991 bytes) 2024-09-13T09:17:44.098 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:44.098 INFO:teuthology.orchestra.run.smithi032.stdout:844 read 0x2d8c446 thru 0x2d8e27d (0x1e38 bytes) 2024-09-13T09:17:44.098 INFO:teuthology.orchestra.run.smithi032.stdout:845 read 0x335c545 thru 0x33618d5 (0x5391 bytes) 2024-09-13T09:17:44.111 INFO:teuthology.orchestra.run.smithi032.stdout:846 write 0xc765379 thru 0xc7749bc (0xf644 bytes) 2024-09-13T09:17:44.118 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:44.114+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:44.130 INFO:teuthology.orchestra.run.smithi032.stdout:848 write 0xeb04c79 thru 0xeb0e58a (0x9912 bytes) 2024-09-13T09:17:44.136 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:44.136 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:44.144 INFO:teuthology.orchestra.run.smithi032.stdout:851 write 0xed092e8 thru 0xed118a5 (0x85be bytes) 2024-09-13T09:17:44.151 INFO:teuthology.orchestra.run.smithi032.stdout:852 write 0x92603b1 thru 0x9269188 (0x8dd8 bytes) 2024-09-13T09:17:44.157 INFO:teuthology.orchestra.run.smithi032.stdout:853 write 0x6e94c27 thru 0x6e9bb2f (0x6f09 bytes) 2024-09-13T09:17:44.163 INFO:teuthology.orchestra.run.smithi032.stdout:855 read 0x278d1ce thru 0x278e443 (0x1276 bytes) 2024-09-13T09:17:44.163 INFO:teuthology.orchestra.run.smithi032.stdout:856 punch from 0x706b7cc to 0x707ac0f, (0xf443 bytes) 2024-09-13T09:17:44.164 INFO:teuthology.orchestra.run.smithi032.stdout:859 write 0x504b817 thru 0x504e379 (0x2b63 bytes) 2024-09-13T09:17:44.169 INFO:teuthology.orchestra.run.smithi032.stdout:860 writesame 0x45a2c7d thru 0x45a7b04 data_size 0x1c(0x4e88 bytes) 2024-09-13T09:17:44.173 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:44.173 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3592 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2920448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.839381+0000", 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3344122199999999, 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0080015380000000007, 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.174 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3592 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839381+0000", 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839381+0000", 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839380+0000", 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839386+0000", 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839405+0000", 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8787000000000002e-05 2024-09-13T09:17:44.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839419+0000", 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3753e-05 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839747+0000", 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032843100000000001 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839760+0000", 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2813000000000001e-05 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.847018+0000", 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0072575909999999999 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.847219+0000", 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201001 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.847382+0000", 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000163399 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3593 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2977792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.839461+0000", 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.334331647, 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083564590000000001, 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.179 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3593 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839461+0000", 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839461+0000", 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839460+0000", 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839470+0000", 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0114e-05 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839491+0000", 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0530000000000002e-05 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839505+0000", 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4525999999999999e-05 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839828+0000", 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032266100000000002 2024-09-13T09:17:44.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839835+0000", 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2699999999999999e-06 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.847419+0000", 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0075842949999999996 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.847619+0000", 2024-09-13T09:17:44.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019961800000000001 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.847818+0000", 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000198839 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3594 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2973696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.839523+0000", 2024-09-13T09:17:44.184 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3342702879999999, 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0087270999999999998, 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3594 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839523+0000", 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839523+0000", 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839522+0000", 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839535+0000", 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2982999999999999e-05 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839553+0000", 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8162999999999999e-05 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839566+0000", 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3716e-05 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839926+0000", 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035991700000000002 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839933+0000", 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.847877+0000", 2024-09-13T09:17:44.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079439130000000004 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.848090+0000", 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000212859 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.848250+0000", 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016019499999999999 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3595 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.839588+0000", 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3342049650000001, 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0091303870000000002, 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3595 2024-09-13T09:17:44.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839588+0000", 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839588+0000", 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839587+0000", 2024-09-13T09:17:44.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839593+0000", 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.443e-06 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839612+0000", 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8204000000000001e-05 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839630+0000", 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8065999999999999e-05 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839982+0000", 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035234699999999998 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839989+0000", 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5599999999999999e-06 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.848288+0000", 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0082990730000000006 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.848488+0000", 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000200325 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.848718+0000", 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000230339 2024-09-13T09:17:44.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3596 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2981888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.839642+0000", 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3341512390000001, 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.009470694, 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3596 2024-09-13T09:17:44.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839642+0000", 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839642+0000", 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839641+0000", 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839647+0000", 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2570000000000004e-06 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839665+0000", 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7693000000000002e-05 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839678+0000", 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3446e-05 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840046+0000", 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036744399999999997 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840052+0000", 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0270000000000001e-06 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.848756+0000", 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.008703967 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.848964+0000", 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000208581 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.849112+0000", 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014823899999999999 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.199 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3597 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.839750+0000", 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3340432790000001, 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0097436980000000003, 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.200 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3597 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839750+0000", 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839750+0000", 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839749+0000", 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839778+0000", 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9196e-05 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839823+0000", 2024-09-13T09:17:44.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5166000000000001e-05 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839842+0000", 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8757e-05 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840112+0000", 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000270678 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840119+0000", 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.08e-06 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.849150+0000", 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.009030873 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.849346+0000", 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019613899999999999 2024-09-13T09:17:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.849493+0000", 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000146849 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3598 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2932736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.839936+0000", 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.333857238, 2024-09-13T09:17:44.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099518269999999999, 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3598 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839936+0000", 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839936+0000", 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839934+0000", 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.839943+0000", 2024-09-13T09:17:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840026+0000", 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.331e-05 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840050+0000", 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4105999999999999e-05 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840159+0000", 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108806 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840168+0000", 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5400000000000001e-06 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.849569+0000", 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094004009999999992 2024-09-13T09:17:44.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.849719+0000", 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150259 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.849888+0000", 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016845200000000001 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3599 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2985984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.840114+0000", 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3336786359999999, 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010106362000000001, 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3599 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840114+0000", 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840114+0000", 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840113+0000", 2024-09-13T09:17:44.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840122+0000", 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5900000000000008e-06 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840190+0000", 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8555999999999999e-05 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840209+0000", 2024-09-13T09:17:44.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8349999999999999e-05 2024-09-13T09:17:44.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840283+0000", 2024-09-13T09:17:44.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4505999999999994e-05 2024-09-13T09:17:44.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840289+0000", 2024-09-13T09:17:44.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:17:44.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.849928+0000", 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0096380960000000005 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.850075+0000", 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014764199999999999 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.850221+0000", 2024-09-13T09:17:44.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000145482 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3600 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2990080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.840231+0000", 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3335622, 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01040903, 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3600 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840231+0000", 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840231+0000", 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840230+0000", 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840237+0000", 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9160000000000001e-06 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840258+0000", 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-05 2024-09-13T09:17:44.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840273+0000", 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4374e-05 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840354+0000", 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1322000000000005e-05 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840364+0000", 2024-09-13T09:17:44.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0293e-05 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.850301+0000", 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099370399999999994 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.850486+0000", 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018494900000000001 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.850640+0000", 2024-09-13T09:17:44.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015358900000000001 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3601 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2936832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.840311+0000", 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3334815609999999, 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010715671, 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.221 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3601 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840311+0000", 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840311+0000", 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840310+0000", 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840333+0000", 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2419999999999999e-05 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840354+0000", 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0896000000000001e-05 2024-09-13T09:17:44.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840370+0000", 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6753999999999999e-05 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840456+0000", 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5681999999999997e-05 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840463+0000", 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5030000000000002e-06 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.850678+0000", 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010215538999999999 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.850876+0000", 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019793499999999999 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.851027+0000", 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015105500000000001 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3602 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.840423+0000", 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3333696449999999, 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010947129, 2024-09-13T09:17:44.226 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3602 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840423+0000", 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840423+0000", 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840422+0000", 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840430+0000", 2024-09-13T09:17:44.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.447e-06 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840458+0000", 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8393000000000001e-05 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840475+0000", 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6742999999999999e-05 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840526+0000", 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1746999999999998e-05 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840534+0000", 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6660000000000008e-06 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.851088+0000", 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010553905000000001 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.851217+0000", 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000128906 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.851370+0000", 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015351199999999999 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3603 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2883584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.840493+0000", 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.333299939, 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011288960000000001, 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3603 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840493+0000", 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840493+0000", 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840492+0000", 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840498+0000", 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.37e-06 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840517+0000", 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8683000000000001e-05 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840531+0000", 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.347e-05 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840618+0000", 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6885999999999994e-05 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840629+0000", 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1963e-05 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.851452+0000", 2024-09-13T09:17:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010822137000000001 2024-09-13T09:17:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.851630+0000", 2024-09-13T09:17:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017793799999999999 2024-09-13T09:17:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.851782+0000", 2024-09-13T09:17:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000152406 2024-09-13T09:17:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3604 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2887680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.840571+0000", 2024-09-13T09:17:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.33322232, 2024-09-13T09:17:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011616395, 2024-09-13T09:17:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3604 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840571+0000", 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840571+0000", 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840570+0000", 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840576+0000", 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5599999999999999e-06 2024-09-13T09:17:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840595+0000", 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8479999999999999e-05 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840608+0000", 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.308e-05 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840700+0000", 2024-09-13T09:17:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2072999999999998e-05 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840714+0000", 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.413e-05 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.851820+0000", 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011105657999999999 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.852036+0000", 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021657100000000001 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.852187+0000", 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150863 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3605 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2945024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.840628+0000", 2024-09-13T09:17:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3331653999999999, 2024-09-13T09:17:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011946602000000001, 2024-09-13T09:17:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3605 2024-09-13T09:17:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840628+0000", 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840628+0000", 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840626+0000", 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840633+0000", 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1469999999999998e-06 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840650+0000", 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7713000000000001e-05 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840667+0000", 2024-09-13T09:17:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6990000000000002e-05 2024-09-13T09:17:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840797+0000", 2024-09-13T09:17:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012954199999999999 2024-09-13T09:17:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840803+0000", 2024-09-13T09:17:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.387e-06 2024-09-13T09:17:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.852224+0000", 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011420539 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.852424+0000", 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000199841 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.852574+0000", 2024-09-13T09:17:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015050299999999999 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3606 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.840680+0000", 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3331126170000001, 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012285622, 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3606 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840680+0000", 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840680+0000", 2024-09-13T09:17:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840679+0000", 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840685+0000", 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8270000000000003e-06 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840702+0000", 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6633000000000002e-05 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840715+0000", 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3016999999999999e-05 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840879+0000", 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016431900000000001 2024-09-13T09:17:44.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840886+0000", 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.852655+0000", 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011768462 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.852801+0000", 2024-09-13T09:17:44.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000146549 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.852966+0000", 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016486199999999999 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3607 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2998272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.840727+0000", 2024-09-13T09:17:44.253 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.333065744, 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01258332, 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3607 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840727+0000", 2024-09-13T09:17:44.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840727+0000", 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840726+0000", 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840732+0000", 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0970000000000002e-06 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840749+0000", 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7367000000000002e-05 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840762+0000", 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:17:44.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840956+0000", 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000193134 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840962+0000", 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3169999999999998e-06 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.853005+0000", 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012043613999999999 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.853156+0000", 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015006899999999999 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.853311+0000", 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493899999999999 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.258 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.259 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.259 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3608 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2895872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.259 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.840806+0000", 2024-09-13T09:17:44.259 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3329870779999999, 2024-09-13T09:17:44.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012915480999999999, 2024-09-13T09:17:44.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3608 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840806+0000", 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840806+0000", 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840805+0000", 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840812+0000", 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:17:44.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840833+0000", 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1083000000000001e-05 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840847+0000", 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4620000000000001e-05 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841025+0000", 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000177842 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841031+0000", 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1929999999999998e-06 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.853397+0000", 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012365265 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.853571+0000", 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000174802 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.853721+0000", 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000149939 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3609 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2953216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.840882+0000", 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.332910788, 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013223074, 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3609 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840882+0000", 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840882+0000", 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840881+0000", 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840888+0000", 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840911+0000", 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2952999999999999e-05 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840925+0000", 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4306e-05 2024-09-13T09:17:44.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841106+0000", 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018106200000000001 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841112+0000", 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.46e-06 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.853759+0000", 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012646833 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.853955+0000", 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019552099999999999 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.854105+0000", 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150369 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3610 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.840958+0000", 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.332835169, 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013546356000000001, 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3610 2024-09-13T09:17:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840958+0000", 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840958+0000", 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840957+0000", 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840963+0000", 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1630000000000001e-06 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840983+0000", 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9757000000000001e-05 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.840996+0000", 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3540000000000001e-05 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841181+0000", 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018456500000000001 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841187+0000", 2024-09-13T09:17:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4969999999999998e-06 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.854183+0000", 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012995398999999999 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.854332+0000", 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014939900000000001 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.854504+0000", 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017198300000000001 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3611 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 2940928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.841036+0000", 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3327572990000001, 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013880801999999999, 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3611 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841036+0000", 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841036+0000", 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841035+0000", 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841041+0000", 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6699999999999997e-06 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841060+0000", 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8845999999999999e-05 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841074+0000", 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4133e-05 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841242+0000", 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000167462 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.841248+0000", 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1469999999999998e-06 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.854590+0000", 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013342557 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.854769+0000", 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017872600000000001 2024-09-13T09:17:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.854916+0000", 2024-09-13T09:17:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014732800000000001 2024-09-13T09:17:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:44.280 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:44.282 INFO:teuthology.orchestra.run.smithi032.stdout:861 read 0xd4e153 thru 0xd51f29 (0x3dd7 bytes) 2024-09-13T09:17:44.282 INFO:teuthology.orchestra.run.smithi032.stdout:863 read 0x7a51916 thru 0x7a51985 (0x70 bytes) 2024-09-13T09:17:44.282 INFO:teuthology.orchestra.run.smithi032.stdout:864 write 0xc7dd70a thru 0xc7e1b39 (0x4430 bytes) 2024-09-13T09:17:44.282 INFO:teuthology.orchestra.run.smithi032.stdout:865 write 0x48aaafc thru 0x48ace80 (0x2385 bytes) 2024-09-13T09:17:44.282 INFO:teuthology.orchestra.run.smithi032.stdout:867 write 0xa2119b thru 0xa2d69f (0xc505 bytes) 2024-09-13T09:17:44.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:17:44.285 INFO:teuthology.orchestra.run.smithi032.stdout:868 write 0xead422c thru 0xeae3e56 (0xfc2b bytes) 2024-09-13T09:17:44.285 INFO:teuthology.orchestra.run.smithi032.stdout:870 write 0xc547f2e thru 0xc548dc9 (0xe9c bytes) 2024-09-13T09:17:44.285 INFO:teuthology.orchestra.run.smithi032.stdout:871 punch from 0x2db2872 to 0x2dc27cc, (0xff5a bytes) 2024-09-13T09:17:44.285 INFO:teuthology.orchestra.run.smithi032.stdout:873 read 0x644766 thru 0x647cc7 (0x3562 bytes) 2024-09-13T09:17:44.285 INFO:teuthology.orchestra.run.smithi032.stdout:875 write 0x5e6aa8 thru 0x5f6894 (0xfded bytes) 2024-09-13T09:17:44.285 INFO:teuthology.orchestra.run.smithi032.stdout:878 read 0x786fb6b thru 0x78715b9 (0x1a4f bytes) 2024-09-13T09:17:44.285 INFO:teuthology.orchestra.run.smithi032.stdout:881 write 0x7d2d95 thru 0x7df608 (0xc874 bytes) 2024-09-13T09:17:44.285 INFO:teuthology.orchestra.run.smithi032.stdout:882 punch from 0x1e4308a to 0x1e4a50f, (0x7485 bytes) 2024-09-13T09:17:44.285 INFO:teuthology.orchestra.run.smithi032.stdout:883 read 0x3aba174 thru 0x3ac9263 (0xf0f0 bytes) 2024-09-13T09:17:44.285 INFO:teuthology.orchestra.run.smithi032.stdout:884 trunc from 0xed118a6 to 0x55f8a29 2024-09-13T09:17:44.286 INFO:teuthology.orchestra.run.smithi032.stdout:886 writesame 0x9a0e586 thru 0x9a199f1 data_size 0x12(0xb46c bytes) 2024-09-13T09:17:44.286 INFO:teuthology.orchestra.run.smithi032.stdout:887 write 0x6ca206c thru 0x6ca7243 (0x51d8 bytes) 2024-09-13T09:17:44.286 INFO:teuthology.orchestra.run.smithi032.stdout:888 punch from 0x358c687 to 0x3599497, (0xce10 bytes) 2024-09-13T09:17:44.286 INFO:teuthology.orchestra.run.smithi032.stdout:890 write 0x43c7cb3 thru 0x43d0fae (0x92fc bytes) 2024-09-13T09:17:44.286 INFO:teuthology.orchestra.run.smithi032.stdout:891 writesame 0xe896de thru 0xe97d22 data_size 0x17(0xe645 bytes) 2024-09-13T09:17:44.286 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:44.278+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:44.291 INFO:teuthology.orchestra.run.smithi032.stdout:893 trunc from 0x9a199f2 to 0x229d3dd 2024-09-13T09:17:44.313 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:44.313 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:44.324 INFO:teuthology.orchestra.run.smithi032.stdout:897 trunc from 0x229d3dd to 0x88cc906 2024-09-13T09:17:44.344 INFO:teuthology.orchestra.run.smithi032.stdout:898 write 0xa91e6e9 thru 0xa921f0f (0x3827 bytes) 2024-09-13T09:17:44.351 INFO:teuthology.orchestra.run.smithi032.stdout:899 read 0x82623e9 thru 0x826cdbf (0xa9d7 bytes) 2024-09-13T09:17:44.352 INFO:teuthology.orchestra.run.smithi032.stdout:900 writesame 0x1386fb7 thru 0x1392909 data_size 0x1(0xb953 bytes) 2024-09-13T09:17:44.382 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:44.378+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:44.390 INFO:teuthology.orchestra.run.smithi032.stdout:901 trunc from 0xa921f10 to 0xa218da4 2024-09-13T09:17:44.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:17:44.411 INFO:teuthology.orchestra.run.smithi032.stdout:902 trunc from 0xa218da4 to 0xb1f9f31 2024-09-13T09:17:44.418 INFO:teuthology.orchestra.run.smithi032.stdout:903 write 0x7066028 thru 0x706ad18 (0x4cf1 bytes) 2024-09-13T09:17:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:44.428 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:44.428 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:44.428 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:44.431 INFO:teuthology.orchestra.run.smithi032.stdout:904 write 0xb61988f thru 0xb621f24 (0x8696 bytes) 2024-09-13T09:17:44.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:17:44.449 INFO:teuthology.orchestra.run.smithi032.stdout:905 write 0xcd9c410 thru 0xcd9d82d (0x141e bytes) 2024-09-13T09:17:44.455 INFO:teuthology.orchestra.run.smithi032.stdout:907 write 0x88bd889 thru 0x88c2480 (0x4bf8 bytes) 2024-09-13T09:17:44.461 INFO:teuthology.orchestra.run.smithi032.stdout:909 read 0xbe66ae thru 0xbe7cfa (0x164d bytes) 2024-09-13T09:17:44.462 INFO:teuthology.orchestra.run.smithi032.stdout:911 writesame 0x957f373 thru 0x95838b3 data_size 0x1(0x4541 bytes) 2024-09-13T09:17:44.483 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:44.478+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:44.486 INFO:teuthology.orchestra.run.smithi032.stdout:912 write 0x52b0c93 thru 0x52bf154 (0xe4c2 bytes) 2024-09-13T09:17:44.491 INFO:teuthology.orchestra.run.smithi032.stdout:913 punch from 0x259b81 to 0x25df8c, (0x440b bytes) 2024-09-13T09:17:44.492 INFO:teuthology.orchestra.run.smithi032.stdout:914 punch from 0x6aa7c81 to 0x6ab50cc, (0xd44b bytes) 2024-09-13T09:17:44.492 INFO:teuthology.orchestra.run.smithi032.stdout:915 read 0x88c2af6 thru 0x88c3299 (0x7a4 bytes) 2024-09-13T09:17:44.492 INFO:teuthology.orchestra.run.smithi032.stdout:916 write 0x284b2db thru 0x2857ec3 (0xcbe9 bytes) 2024-09-13T09:17:44.497 INFO:teuthology.orchestra.run.smithi032.stdout:917 punch from 0xc9562b1 to 0xc9578c2, (0x1611 bytes) 2024-09-13T09:17:44.497 INFO:teuthology.orchestra.run.smithi032.stdout:918 write 0x20ce59f thru 0x20d5be4 (0x7646 bytes) 2024-09-13T09:17:44.502 INFO:teuthology.orchestra.run.smithi032.stdout:919 read 0xa6bb9d5 thru 0xa6bd4a9 (0x1ad5 bytes) 2024-09-13T09:17:44.502 INFO:teuthology.orchestra.run.smithi032.stdout:925 writesame 0x1a91cc8 thru 0x1a955c3 data_size 0xe(0x38fc bytes) 2024-09-13T09:17:44.508 INFO:teuthology.orchestra.run.smithi032.stdout:926 write 0xc74730d thru 0xc74b212 (0x3f06 bytes) 2024-09-13T09:17:44.514 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:44.514 INFO:teuthology.orchestra.run.smithi032.stdout:929 write 0xabe9366 thru 0xabeea7a (0x5715 bytes) 2024-09-13T09:17:44.520 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:44.520 INFO:teuthology.orchestra.run.smithi032.stdout:933 writesame 0x97f7085 thru 0x97fc5e1 data_size 0xd(0x555d bytes) 2024-09-13T09:17:44.527 INFO:teuthology.orchestra.run.smithi032.stdout:934 punch from 0x3aea8a9 to 0x3af9aa5, (0xf1fc bytes) 2024-09-13T09:17:44.528 INFO:teuthology.orchestra.run.smithi032.stdout:935 flatten 2024-09-13T09:17:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:44.583 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:44.584 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:44.580+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:44.600 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:44.617 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:44.618 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:44.618 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:44.619 INFO:teuthology.orchestra.run.smithi032.stdout:936 punch from 0x1f444e1 to 0x1f5432a, (0xfe49 bytes) 2024-09-13T09:17:44.619 INFO:teuthology.orchestra.run.smithi032.stdout:937 read 0x4d97292 thru 0x4da40da (0xce49 bytes) 2024-09-13T09:17:44.619 INFO:teuthology.orchestra.run.smithi032.stdout:938 read 0x7a237cd thru 0x7a2beab (0x86df bytes) 2024-09-13T09:17:44.619 INFO:teuthology.orchestra.run.smithi032.stdout:940 write 0x384183c thru 0x38452dc (0x3aa1 bytes) 2024-09-13T09:17:44.624 INFO:teuthology.orchestra.run.smithi032.stdout:941 clone 5 order 21 su 65536 sc 14 2024-09-13T09:17:44.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:17:44.634 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:44.655 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:44.675 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:44.685 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:44.680+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:44.710 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:44.744 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:44.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:17:44.787 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:44.782+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:44.814 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:17:44.877 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:17:44.877 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:17:44.877 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-13T09:17:44.878 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:17:44.878 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:17:44.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-13T09:17:44.926 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:44.924+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:44.936 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:44.936 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:44.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:17:45.002 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:45.000+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:45.006 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:17:45.006 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:45.089 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:45.082+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:45.112 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-13T09:17:45.120 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:45.120 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:45.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:17:45.191 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:45.188+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:45.291 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:45.286+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:45.319 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:45.319 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:45.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:17:45.392 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:45.388+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:45.399 INFO:teuthology.orchestra.run.smithi032.stdout:943 read 0x4a3b7dc thru 0x4a3b824 (0x49 bytes) 2024-09-13T09:17:45.413 INFO:teuthology.orchestra.run.smithi032.stdout:947 writesame 0xd796499 thru 0xd79bdba data_size 0x3(0x5922 bytes) 2024-09-13T09:17:45.415 INFO:teuthology.orchestra.run.smithi032.stdout:948 write 0x9090b75 thru 0x909ce07 (0xc293 bytes) 2024-09-13T09:17:45.419 INFO:teuthology.orchestra.run.smithi032.stdout:950 trunc from 0xd79bdbb to 0xd4ded38 2024-09-13T09:17:45.438 INFO:teuthology.orchestra.run.smithi032.stdout:951 read 0xcee42db thru 0xceea89d (0x65c3 bytes) 2024-09-13T09:17:45.450 INFO:teuthology.orchestra.run.smithi032.stdout:952 trunc from 0xd4ded38 to 0x3cae419 2024-09-13T09:17:45.469 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:45.469 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:45.486 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:17:45.493 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:45.488+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:45.510 INFO:teuthology.orchestra.run.smithi032.stdout:954 write 0xa25978a thru 0xa267c1a (0xe491 bytes) 2024-09-13T09:17:45.516 INFO:teuthology.orchestra.run.smithi032.stdout:957 read 0x85f89dd thru 0x8601167 (0x878b bytes) 2024-09-13T09:17:45.516 INFO:teuthology.orchestra.run.smithi032.stdout:958 read 0x8dcb3b3 thru 0x8dcee95 (0x3ae3 bytes) 2024-09-13T09:17:45.517 INFO:teuthology.orchestra.run.smithi032.stdout:959 read 0x798116e thru 0x7983567 (0x23fa bytes) 2024-09-13T09:17:45.517 INFO:teuthology.orchestra.run.smithi032.stdout:960 punch from 0x68a3ee8 to 0x68aced8, (0x8ff0 bytes) 2024-09-13T09:17:45.517 INFO:teuthology.orchestra.run.smithi032.stdout:963 read 0x65167a8 thru 0x6524fd8 (0xe831 bytes) 2024-09-13T09:17:45.517 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:45.533 INFO:teuthology.orchestra.run.smithi032.stdout:967 writesame 0xeb207ae thru 0xeb24eca data_size 0xb(0x471d bytes) 2024-09-13T09:17:45.533 INFO:teuthology.orchestra.run.smithi032.stdout:968 read 0x4c078f3 thru 0x4c07d85 (0x493 bytes) 2024-09-13T09:17:45.533 INFO:teuthology.orchestra.run.smithi032.stdout:969 punch from 0xdd9149b to 0xdd9648e, (0x4ff3 bytes) 2024-09-13T09:17:45.534 INFO:teuthology.orchestra.run.smithi032.stdout:971 writesame 0x124efa9 thru 0x125d827 data_size 0x1(0xe87f bytes) 2024-09-13T09:17:45.576 INFO:teuthology.orchestra.run.smithi032.stdout:974 read 0x574df9 thru 0x575d19 (0xf21 bytes) 2024-09-13T09:17:45.578 INFO:teuthology.orchestra.run.smithi032.stdout:976 writesame 0x663ddae thru 0x664b6bc data_size 0x1(0xd90f bytes) 2024-09-13T09:17:45.594 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:45.592+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:45.624 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:45.624 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:45.635 INFO:teuthology.orchestra.run.smithi032.stdout:978 write 0x2f37ae thru 0x301675 (0xdec8 bytes) 2024-09-13T09:17:45.640 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:17:45.646 INFO:teuthology.orchestra.run.smithi032.stdout:979 write 0xcf99243 thru 0xcfa6561 (0xd31f bytes) 2024-09-13T09:17:45.654 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:45.654 INFO:teuthology.orchestra.run.smithi032.stdout:982 write 0xaf8806d thru 0xaf8af1d (0x2eb1 bytes) 2024-09-13T09:17:45.675 INFO:teuthology.orchestra.run.smithi032.stdout:983 trunc from 0xeb24ecb to 0x8ef37bf 2024-09-13T09:17:45.694 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:45.690+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:45.696 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:45.709 INFO:teuthology.orchestra.run.smithi032.stdout:985 write 0x9bb314a thru 0x9bbc5ea (0x94a1 bytes) 2024-09-13T09:17:45.716 INFO:teuthology.orchestra.run.smithi032.stdout:986 read 0x641040d thru 0x6419a30 (0x9624 bytes) 2024-09-13T09:17:45.716 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:45.716 INFO:teuthology.orchestra.run.smithi032.stdout:988 read 0x4ba5b2 thru 0x4bd9b5 (0x3404 bytes) 2024-09-13T09:17:45.717 INFO:teuthology.orchestra.run.smithi032.stdout:989 write 0x1e98afe thru 0x1ea813c (0xf63f bytes) 2024-09-13T09:17:45.741 INFO:teuthology.orchestra.run.smithi032.stdout:990 write 0xa744c8e thru 0xa7485ae (0x3921 bytes) 2024-09-13T09:17:45.748 INFO:teuthology.orchestra.run.smithi032.stdout:992 read 0x841abba thru 0x84231cb (0x8612 bytes) 2024-09-13T09:17:45.748 INFO:teuthology.orchestra.run.smithi032.stdout:993 read 0x9587cc7 thru 0x95954bc (0xd7f6 bytes) 2024-09-13T09:17:45.749 INFO:teuthology.orchestra.run.smithi032.stdout:994 writesame 0x7c636a5 thru 0x7c68247 data_size 0x11(0x4ba3 bytes) 2024-09-13T09:17:45.756 INFO:teuthology.orchestra.run.smithi032.stdout:995 read 0x221e6e6 thru 0x222aab2 (0xc3cd bytes) 2024-09-13T09:17:45.758 INFO:teuthology.orchestra.run.smithi032.stdout:996 trunc from 0xa7485af to 0xa4f9aa9 2024-09-13T09:17:45.773 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:45.773 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:45.774 INFO:teuthology.orchestra.run.smithi032.stdout:998 writesame 0x17353d3 thru 0x173c045 data_size 0x1(0x6c73 bytes) 2024-09-13T09:17:45.789 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:17:45.796 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:45.792+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:45.796 INFO:teuthology.orchestra.run.smithi032.stdout:999 write 0x167e2e5 thru 0x168a0f7 (0xbe13 bytes) 2024-09-13T09:17:45.809 INFO:teuthology.orchestra.run.smithi032.stdout:1000 write 0x585dff2 thru 0x586d8f8 (0xf907 bytes) 2024-09-13T09:17:45.829 INFO:teuthology.orchestra.run.smithi032.stdout:1001 trunc from 0xa4f9aa9 to 0x622906f 2024-09-13T09:17:45.871 INFO:teuthology.orchestra.run.smithi032.stdout:1002 trunc from 0x622906f to 0xd3d6b3a 2024-09-13T09:17:45.882 INFO:teuthology.orchestra.run.smithi032.stdout:1003 write 0xb949195 thru 0xb95374a (0xa5b6 bytes) 2024-09-13T09:17:45.890 INFO:teuthology.orchestra.run.smithi032.stdout:1004 read 0xa0e1b7d thru 0xa0ed421 (0xb8a5 bytes) 2024-09-13T09:17:45.891 INFO:teuthology.orchestra.run.smithi032.stdout:1008 write 0x558198f thru 0x558df62 (0xc5d4 bytes) 2024-09-13T09:17:45.896 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:45.892+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:45.897 INFO:teuthology.orchestra.run.smithi032.stdout:1009 read 0x80305e6 thru 0x80369f0 (0x640b bytes) 2024-09-13T09:17:45.897 INFO:teuthology.orchestra.run.smithi032.stdout:1010 read 0x7cd09f7 thru 0x7cd7baa (0x71b4 bytes) 2024-09-13T09:17:45.897 INFO:teuthology.orchestra.run.smithi032.stdout:1011 punch from 0x8a7d2f5 to 0x8a879ee, (0xa6f9 bytes) 2024-09-13T09:17:45.924 INFO:teuthology.orchestra.run.smithi032.stdout:1012 trunc from 0xd3d6b3a to 0x3aa92be 2024-09-13T09:17:45.956 INFO:teuthology.orchestra.run.smithi032.stdout:1013 write 0x2643548 thru 0x264b89b (0x8354 bytes) 2024-09-13T09:17:45.967 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:45.968 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:45.982 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:17:45.994 INFO:teuthology.orchestra.run.smithi032.stdout:1014 write 0xae7b181 thru 0xae8659f (0xb41f bytes) 2024-09-13T09:17:45.997 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:45.992+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:46.003 INFO:teuthology.orchestra.run.smithi032.stdout:1016 write 0x51c89db thru 0x51d6a44 (0xe06a bytes) 2024-09-13T09:17:46.005 INFO:teuthology.orchestra.run.smithi032.stdout:1017 read 0x6b6ec82 thru 0x6b70f3e (0x22bd bytes) 2024-09-13T09:17:46.018 INFO:teuthology.orchestra.run.smithi032.stdout:1018 write 0xc3e1cfe thru 0xc3ecc5a (0xaf5d bytes) 2024-09-13T09:17:46.023 INFO:teuthology.orchestra.run.smithi032.stdout:1022 punch from 0x222db5 to 0x22c50c, (0x9757 bytes) 2024-09-13T09:17:46.023 INFO:teuthology.orchestra.run.smithi032.stdout:1023 punch from 0xc110937 to 0xc11c828, (0xbef1 bytes) 2024-09-13T09:17:46.023 INFO:teuthology.orchestra.run.smithi032.stdout:1024 write 0x8369db thru 0x8372c1 (0x8e7 bytes) 2024-09-13T09:17:46.026 INFO:teuthology.orchestra.run.smithi032.stdout:1025 read 0xb46e654 thru 0xb479d1e (0xb6cb bytes) 2024-09-13T09:17:46.033 INFO:teuthology.orchestra.run.smithi032.stdout:1026 trunc from 0xc3ecc5b to 0x9ed5d7a 2024-09-13T09:17:46.057 INFO:teuthology.orchestra.run.smithi032.stdout:1027 trunc from 0x9ed5d7a to 0x82749cb 2024-09-13T09:17:46.067 INFO:teuthology.orchestra.run.smithi032.stdout:1028 punch from 0xe230c3 to 0xe27533, (0x4470 bytes) 2024-09-13T09:17:46.068 INFO:teuthology.orchestra.run.smithi032.stdout:1029 trunc from 0x82749cb to 0x79041dd 2024-09-13T09:17:46.091 INFO:teuthology.orchestra.run.smithi032.stdout:1030 trunc from 0x79041dd to 0xc201d00 2024-09-13T09:17:46.098 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:46.094+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:46.102 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:46.106 INFO:teuthology.orchestra.run.smithi032.stdout:1032 trunc from 0xc201d00 to 0xadfb236 2024-09-13T09:17:46.117 INFO:teuthology.orchestra.run.smithi032.stdout:1034 read 0x7228fc6 thru 0x7236855 (0xd890 bytes) 2024-09-13T09:17:46.117 INFO:teuthology.orchestra.run.smithi032.stdout:1036 read 0x1dc8269 thru 0x1dcfe3c (0x7bd4 bytes) 2024-09-13T09:17:46.118 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:46.118 INFO:teuthology.orchestra.run.smithi032.stdout:1038 write 0x7c46999 thru 0x7c557cb (0xee33 bytes) 2024-09-13T09:17:46.119 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:46.119 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:46.127 INFO:teuthology.orchestra.run.smithi032.stdout:1041 read 0x415a30b thru 0x41618e2 (0x75d8 bytes) 2024-09-13T09:17:46.128 INFO:teuthology.orchestra.run.smithi032.stdout:1042 punch from 0x3015fc6 to 0x3021a86, (0xbac0 bytes) 2024-09-13T09:17:46.129 INFO:teuthology.orchestra.run.smithi032.stdout:1043 write 0x3d37d56 thru 0x3d4204e (0xa2f9 bytes) 2024-09-13T09:17:46.132 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:46.132 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:46.132 INFO:teuthology.orchestra.run.smithi032.stdout:1046 write 0x1a9429b thru 0x1a9bbbc (0x7922 bytes) 2024-09-13T09:17:46.134 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:17:46.135 INFO:teuthology.orchestra.run.smithi032.stdout:1047 read 0x54fbf0e thru 0x550528c (0x937f bytes) 2024-09-13T09:17:46.135 INFO:teuthology.orchestra.run.smithi032.stdout:1048 writesame 0xa759108 thru 0xa76376d data_size 0x3b(0xa666 bytes) 2024-09-13T09:17:46.144 INFO:teuthology.orchestra.run.smithi032.stdout:1049 write 0x2311013 thru 0x2314898 (0x3886 bytes) 2024-09-13T09:17:46.152 INFO:teuthology.orchestra.run.smithi032.stdout:1050 read 0xaa7b81b thru 0xaa7b8d3 (0xb9 bytes) 2024-09-13T09:17:46.152 INFO:teuthology.orchestra.run.smithi032.stdout:1051 read 0x16f348d thru 0x16f856b (0x50df bytes) 2024-09-13T09:17:46.153 INFO:teuthology.orchestra.run.smithi032.stdout:1052 clone 6 order 24 su 262144 sc 6 2024-09-13T09:17:46.200 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:46.196+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:46.301 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:46.294+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:46.305 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:46.305 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:46.321 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:17:46.406 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:46.402+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:46.489 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:46.489 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:46.502 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:46.500+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:46.503 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:17:46.603 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:46.604+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:46.672 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:46.672 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:46.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:17:46.734 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:46.730+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:46.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:17:46.805 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:46.798+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:46.811 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:46.812 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:46.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:17:46.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:17:46.907 INFO:teuthology.orchestra.run.smithi032.stdout:leaving image image_client.0-clone5 intact 2024-09-13T09:17:46.917 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:46.914+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:46.926 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:46.927 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:46.944 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:46.944 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:46.961 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:46.961 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:46.979 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:46.979 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:46.999 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:47.000 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:47.005 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:47.006 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:47.006 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:47.004+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:47.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:17:47.025 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:47.045 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:47.064 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:47.064 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:47.078 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-13T09:17:47.110 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:47.106+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:47.202 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:17:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:17:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:17:47.264 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:47.266 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:17:47.266 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:47.262+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:47.367 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:47.362+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:47.442 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:17:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:17:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:17:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.046338820999999, 2024-09-13T09:17:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077478195999999999, 2024-09-13T09:17:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392636+0000", 2024-09-13T09:17:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392651+0000", 2024-09-13T09:17:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:17:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392674+0000", 2024-09-13T09:17:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.264e-05 2024-09-13T09:17:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392691+0000", 2024-09-13T09:17:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7203000000000001e-05 2024-09-13T09:17:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392780+0000", 2024-09-13T09:17:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8671999999999994e-05 2024-09-13T09:17:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:17:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392794+0000", 2024-09-13T09:17:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-13T09:17:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.466664+0000", 2024-09-13T09:17:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073869913999999995 2024-09-13T09:17:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467139+0000", 2024-09-13T09:17:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047488599999999999 2024-09-13T09:17:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467664+0000", 2024-09-13T09:17:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052530600000000002 2024-09-13T09:17:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469125+0000", 2024-09-13T09:17:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001460432 2024-09-13T09:17:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469847+0000", 2024-09-13T09:17:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072197799999999996 2024-09-13T09:17:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470006+0000", 2024-09-13T09:17:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015955900000000001 2024-09-13T09:17:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470116+0000", 2024-09-13T09:17:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109836 2024-09-13T09:17:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5213 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538498+0000", 2024-09-13T09:17:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.9004789469999999, 2024-09-13T09:17:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049914764, 2024-09-13T09:17:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5213 2024-09-13T09:17:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538498+0000", 2024-09-13T09:17:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538498+0000", 2024-09-13T09:17:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538497+0000", 2024-09-13T09:17:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538506+0000", 2024-09-13T09:17:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2129999999999997e-06 2024-09-13T09:17:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538515+0000", 2024-09-13T09:17:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:17:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538522+0000", 2024-09-13T09:17:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.007e-06 2024-09-13T09:17:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588044+0000", 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049521823999999999 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588264+0000", 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022035199999999999 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588413+0000", 2024-09-13T09:17:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014863499999999999 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5214 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538528+0000", 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.9004490469999999, 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050251938000000003, 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5214 2024-09-13T09:17:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538528+0000", 2024-09-13T09:17:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538528+0000", 2024-09-13T09:17:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538527+0000", 2024-09-13T09:17:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538530+0000", 2024-09-13T09:17:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2499999999999998e-06 2024-09-13T09:17:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538539+0000", 2024-09-13T09:17:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.14e-06 2024-09-13T09:17:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538562+0000", 2024-09-13T09:17:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3107000000000001e-05 2024-09-13T09:17:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588494+0000", 2024-09-13T09:17:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049932257000000001 2024-09-13T09:17:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588615+0000", 2024-09-13T09:17:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120932 2024-09-13T09:17:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588780+0000", 2024-09-13T09:17:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000164869 2024-09-13T09:17:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5215 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538572+0000", 2024-09-13T09:17:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.9004051610000001, 2024-09-13T09:17:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050803344, 2024-09-13T09:17:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5215 2024-09-13T09:17:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538572+0000", 2024-09-13T09:17:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538572+0000", 2024-09-13T09:17:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538571+0000", 2024-09-13T09:17:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538575+0000", 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538585+0000", 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8400000000000007e-06 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538595+0000", 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0346000000000001e-05 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588863+0000", 2024-09-13T09:17:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050268238 2024-09-13T09:17:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.589092+0000", 2024-09-13T09:17:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022874500000000001 2024-09-13T09:17:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.589375+0000", 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028307799999999998 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5216 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 450560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538638+0000", 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.9003386349999998, 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051242401, 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5216 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538638+0000", 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538638+0000", 2024-09-13T09:17:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538637+0000", 2024-09-13T09:17:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538644+0000", 2024-09-13T09:17:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:17:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538680+0000", 2024-09-13T09:17:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.663e-05 2024-09-13T09:17:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538697+0000", 2024-09-13T09:17:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7033000000000001e-05 2024-09-13T09:17:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.589442+0000", 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050745014999999997 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.589619+0000", 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000176338 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.589881+0000", 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000261795 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5217 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538710+0000", 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.9002667249999998, 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051702696999999999, 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5217 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538710+0000", 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538710+0000", 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538709+0000", 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538719+0000", 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4129999999999995e-06 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538738+0000", 2024-09-13T09:17:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9196000000000001e-05 2024-09-13T09:17:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538755+0000", 2024-09-13T09:17:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7413000000000001e-05 2024-09-13T09:17:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.589952+0000", 2024-09-13T09:17:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051197222000000001 2024-09-13T09:17:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590167+0000", 2024-09-13T09:17:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000214444 2024-09-13T09:17:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590413+0000", 2024-09-13T09:17:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024592199999999998 2024-09-13T09:17:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5218 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538768+0000", 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.9002092290000001, 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052032464, 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5218 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538768+0000", 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538768+0000", 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538767+0000", 2024-09-13T09:17:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538773+0000", 2024-09-13T09:17:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1e-06 2024-09-13T09:17:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538790+0000", 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7266999999999999e-05 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538803+0000", 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3179999999999999e-05 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590503+0000", 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051700044000000001 2024-09-13T09:17:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590645+0000", 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000141435 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590800+0000", 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015541200000000001 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5219 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538814+0000", 2024-09-13T09:17:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.9001627559999998, 2024-09-13T09:17:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052311642999999998, 2024-09-13T09:17:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5219 2024-09-13T09:17:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538814+0000", 2024-09-13T09:17:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538814+0000", 2024-09-13T09:17:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538813+0000", 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538819+0000", 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538836+0000", 2024-09-13T09:17:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:17:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538849+0000", 2024-09-13T09:17:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2896e-05 2024-09-13T09:17:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590838+0000", 2024-09-13T09:17:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051988922 2024-09-13T09:17:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590954+0000", 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116422 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591126+0000", 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017145600000000001 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5220 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538860+0000", 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.9001165599999998, 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052651584000000001, 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5220 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538860+0000", 2024-09-13T09:17:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538860+0000", 2024-09-13T09:17:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538859+0000", 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538865+0000", 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4639999999999997e-06 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538884+0000", 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8963000000000001e-05 2024-09-13T09:17:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538897+0000", 2024-09-13T09:17:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3123e-05 2024-09-13T09:17:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591215+0000", 2024-09-13T09:17:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052317782 2024-09-13T09:17:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591355+0000", 2024-09-13T09:17:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014062600000000001 2024-09-13T09:17:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591512+0000", 2024-09-13T09:17:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000156606 2024-09-13T09:17:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5221 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538908+0000", 2024-09-13T09:17:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.9000687439999999, 2024-09-13T09:17:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052944665000000002, 2024-09-13T09:17:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5221 2024-09-13T09:17:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538908+0000", 2024-09-13T09:17:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538908+0000", 2024-09-13T09:17:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538907+0000", 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538913+0000", 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7429999999999999e-06 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538929+0000", 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6609999999999999e-05 2024-09-13T09:17:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538948+0000", 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8697000000000001e-05 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591582+0000", 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052633390000000002 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591709+0000", 2024-09-13T09:17:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012750899999999999 2024-09-13T09:17:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591853+0000", 2024-09-13T09:17:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014367899999999999 2024-09-13T09:17:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5222 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 458752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538961+0000", 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.9000162400000002, 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053306521000000003, 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5222 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538961+0000", 2024-09-13T09:17:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538961+0000", 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538960+0000", 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538965+0000", 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7230000000000001e-06 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539003+0000", 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7323000000000002e-05 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539018+0000", 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5492999999999999e-05 2024-09-13T09:17:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591937+0000", 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052918924999999999 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592115+0000", 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000178388 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592267+0000", 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015165600000000001 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5223 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539031+0000", 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.8999456850000001, 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053594032999999999, 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5223 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539031+0000", 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539031+0000", 2024-09-13T09:17:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539030+0000", 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539037+0000", 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6359999999999997e-06 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539054+0000", 2024-09-13T09:17:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7354e-05 2024-09-13T09:17:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539068+0000", 2024-09-13T09:17:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3633e-05 2024-09-13T09:17:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592349+0000", 2024-09-13T09:17:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053281308999999999 2024-09-13T09:17:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592484+0000", 2024-09-13T09:17:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134812 2024-09-13T09:17:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592625+0000", 2024-09-13T09:17:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000141345 2024-09-13T09:17:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5224 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539083+0000", 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.8998939149999998, 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0539636, 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5224 2024-09-13T09:17:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539083+0000", 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539083+0000", 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539082+0000", 2024-09-13T09:17:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539088+0000", 2024-09-13T09:17:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:17:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539106+0000", 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7543000000000001e-05 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539119+0000", 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3363000000000001e-05 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592700+0000", 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053580790000000003 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592907+0000", 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000207161 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593046+0000", 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013960599999999999 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5225 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539130+0000", 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.8998469490000001, 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054364970999999998, 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5225 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539130+0000", 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539130+0000", 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539129+0000", 2024-09-13T09:17:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539149+0000", 2024-09-13T09:17:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0095999999999999e-05 2024-09-13T09:17:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539167+0000", 2024-09-13T09:17:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8284e-05 2024-09-13T09:17:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539183+0000", 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5645999999999999e-05 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593117+0000", 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053933794 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593339+0000", 2024-09-13T09:17:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022261500000000001 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593495+0000", 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015542900000000001 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5226 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539196+0000", 2024-09-13T09:17:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.8997809119999998, 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054730256999999997, 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5226 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539196+0000", 2024-09-13T09:17:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539196+0000", 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539195+0000", 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539201+0000", 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3600000000000001e-06 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539239+0000", 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539259+0000", 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9656999999999998e-05 2024-09-13T09:17:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593577+0000", 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054317773999999999 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593748+0000", 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017108499999999999 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593926+0000", 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017829200000000001 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5227 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539286+0000", 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.8996910969999998, 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055249030999999997, 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5227 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539286+0000", 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539286+0000", 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539285+0000", 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539290+0000", 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1000000000000003e-06 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539312+0000", 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2276000000000002e-05 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539321+0000", 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9099999999999994e-06 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.594016+0000", 2024-09-13T09:17:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054694995000000003 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.594252+0000", 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023619799999999999 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.594535+0000", 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028231799999999999 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5228 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539328+0000", 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.8996482829999999, 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055582577000000001, 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5228 2024-09-13T09:17:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539328+0000", 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539328+0000", 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539328+0000", 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539332+0000", 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539341+0000", 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5629999999999998e-06 2024-09-13T09:17:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6299999999999998e-06 2024-09-13T09:17:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.594616+0000", 2024-09-13T09:17:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055267509999999999 2024-09-13T09:17:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.594733+0000", 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011629399999999999 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.594911+0000", 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000178567 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5229 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539355+0000", 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.8996221470000001, 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.056017574000000001, 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5229 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539355+0000", 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539355+0000", 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539354+0000", 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539358+0000", 2024-09-13T09:17:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3470000000000001e-06 2024-09-13T09:17:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539367+0000", 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6230000000000003e-06 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539376+0000", 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7469999999999994e-06 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595014+0000", 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055638340000000001 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595197+0000", 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018309199999999999 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595372+0000", 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017503500000000001 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5230 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539384+0000", 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.8995925009999999, 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.056529714000000002, 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5230 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539384+0000", 2024-09-13T09:17:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539384+0000", 2024-09-13T09:17:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539384+0000", 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539387+0000", 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2669999999999998e-06 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539395+0000", 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2859999999999999e-06 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539402+0000", 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595457+0000", 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.056054567999999999 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595675+0000", 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000218128 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595914+0000", 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000239328 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5231 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539412+0000", 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.8995652540000001, 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.057015146000000003, 2024-09-13T09:17:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5231 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539412+0000", 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539412+0000", 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539411+0000", 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539414+0000", 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-13T09:17:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539423+0000", 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4200000000000007e-06 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539430+0000", 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1330000000000001e-06 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595979+0000", 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.056549006999999998 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.596198+0000", 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219292 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.596427+0000", 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000228691 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:47.548 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:47.549 INFO:teuthology.orchestra.run.smithi032.stdout:1053 trunc from 0xadfb236 to 0x4f49ed8 2024-09-13T09:17:47.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:17:47.550 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:47.544+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:47.602 INFO:teuthology.orchestra.run.smithi032.stdout:1054 write 0x283aa45 thru 0x2842717 (0x7cd3 bytes) 2024-09-13T09:17:47.604 INFO:teuthology.orchestra.run.smithi032.stdout:1055 punch from 0x4c9a133 to 0x4c9a99e, (0x86b bytes) 2024-09-13T09:17:47.605 INFO:teuthology.orchestra.run.smithi032.stdout:1056 write 0x45f3000 thru 0x46019e0 (0xe9e1 bytes) 2024-09-13T09:17:47.621 INFO:teuthology.orchestra.run.smithi032.stdout:1057 writesame 0x8047b0c thru 0x8055a5a data_size 0x1(0xdf4f bytes) 2024-09-13T09:17:47.634 INFO:teuthology.orchestra.run.smithi032.stdout:1059 write 0xb3af9d1 thru 0xb3b3cfa (0x432a bytes) 2024-09-13T09:17:47.640 INFO:teuthology.orchestra.run.smithi032.stdout:1060 read 0x1bd80e2 thru 0x1bdcd93 (0x4cb2 bytes) 2024-09-13T09:17:47.650 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:47.646+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:47.651 INFO:teuthology.orchestra.run.smithi032.stdout:1061 writesame 0xb7d1fba thru 0xb7db85a data_size 0x1(0x98a1 bytes) 2024-09-13T09:17:47.651 INFO:teuthology.orchestra.run.smithi032.stdout:1062 trunc from 0xb7db85b to 0xb6c8ad7 2024-09-13T09:17:47.663 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:47.663 INFO:teuthology.orchestra.run.smithi032.stdout:1065 write 0x6a05027 thru 0x6a0de53 (0x8e2d bytes) 2024-09-13T09:17:47.666 INFO:teuthology.orchestra.run.smithi032.stdout:1067 read 0x619aaf4 thru 0x61a5933 (0xae40 bytes) 2024-09-13T09:17:47.666 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:47.676 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:47.676 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:47.676 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:47.676 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:47.683 INFO:teuthology.orchestra.run.smithi032.stdout:1069 write 0xe3223ed thru 0xe3254e8 (0x30fc bytes) 2024-09-13T09:17:47.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:17:47.690 INFO:teuthology.orchestra.run.smithi032.stdout:1070 trunc from 0xe3254e9 to 0xeaa3e18 2024-09-13T09:17:47.701 INFO:teuthology.orchestra.run.smithi032.stdout:1071 clone 7 order 18 su 131072 sc 4 2024-09-13T09:17:47.752 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:47.746+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:47.805 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:47.806 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:47.820 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:17:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:17:47.853 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:47.846+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:47.954 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:47.952+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:47.958 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:47.958 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:47.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:47.958 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:47.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.958 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3:ed699f0f:::rbd_data.118591b29fad.000000000000008c:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:47.958 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:17:47.958 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.0949331940000002, 2024-09-13T09:17:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94878089700000001, 2024-09-13T09:17:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:17:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:47.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:17:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810654+0000", 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94834837599999999 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810755+0000", 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100987 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810828+0000", 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2751999999999997e-05 2024-09-13T09:17:47.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4102 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973478+0000", 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.983502171, 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023485389999999998, 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:47.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4102 2024-09-13T09:17:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973478+0000", 2024-09-13T09:17:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973478+0000", 2024-09-13T09:17:47.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973478+0000", 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973480+0000", 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973486+0000", 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.156e-06 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973491+0000", 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7790000000000003e-06 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.996766+0000", 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023274522999999998 2024-09-13T09:17:47.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.996862+0000", 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6371000000000003e-05 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.996964+0000", 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010140399999999999 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4103 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973495+0000", 2024-09-13T09:17:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9834849500000002, 2024-09-13T09:17:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023681364999999999, 2024-09-13T09:17:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4103 2024-09-13T09:17:47.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973495+0000", 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973495+0000", 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973495+0000", 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973497+0000", 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3190000000000002e-06 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973503+0000", 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0909999999999998e-06 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973508+0000", 2024-09-13T09:17:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-13T09:17:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.996995+0000", 2024-09-13T09:17:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023486717000000001 2024-09-13T09:17:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:47.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997080+0000", 2024-09-13T09:17:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5500999999999993e-05 2024-09-13T09:17:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997177+0000", 2024-09-13T09:17:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6325999999999999e-05 2024-09-13T09:17:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.973 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4104 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9834554359999998, 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023862547000000001, 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4104 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973524+0000", 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973527+0000", 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1200000000000002e-06 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973535+0000", 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2479999999999999e-06 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973548+0000", 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.365e-05 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997205+0000", 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023657088999999999 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997289+0000", 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3801000000000006e-05 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997387+0000", 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8317000000000001e-05 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4105 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9834269190000002, 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024049968000000001, 2024-09-13T09:17:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4105 2024-09-13T09:17:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:17:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:17:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:17:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973556+0000", 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7420000000000002e-06 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973564+0000", 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973569+0000", 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-13T09:17:47.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997418+0000", 2024-09-13T09:17:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023849052999999999 2024-09-13T09:17:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997504+0000", 2024-09-13T09:17:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6075999999999994e-05 2024-09-13T09:17:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997603+0000", 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9455000000000002e-05 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4106 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9834073280000002, 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024364336, 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:47.983 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4106 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973575+0000", 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.322e-06 2024-09-13T09:17:47.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973594+0000", 2024-09-13T09:17:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9031000000000001e-05 2024-09-13T09:17:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973601+0000", 2024-09-13T09:17:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5300000000000002e-06 2024-09-13T09:17:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997648+0000", 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024047731999999999 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997748+0000", 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9807000000000002e-05 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997937+0000", 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018924299999999999 2024-09-13T09:17:47.987 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4107 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:17:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9833748839999998, 2024-09-13T09:17:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542690999999998, 2024-09-13T09:17:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4107 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:17:47.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973614+0000", 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:47.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973621+0000", 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0110000000000003e-06 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973626+0000", 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0320000000000003e-06 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997979+0000", 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024353063000000001 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998059+0000", 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9221000000000006e-05 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998148+0000", 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9506999999999995e-05 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4108 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9833492760000002, 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024717223999999999, 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4108 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:47.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:17:47.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973633+0000", 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2089999999999999e-06 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973639+0000", 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1469999999999998e-06 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973644+0000", 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5310000000000003e-06 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998175+0000", 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024531826999999999 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998257+0000", 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1668999999999993e-05 2024-09-13T09:17:47.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998348+0000", 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1184e-05 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4109 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9833297449999998, 2024-09-13T09:17:47.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024935052999999999, 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4109 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973650+0000", 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:47.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973653+0000", 2024-09-13T09:17:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.328e-06 2024-09-13T09:17:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973659+0000", 2024-09-13T09:17:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3269999999999997e-06 2024-09-13T09:17:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973664+0000", 2024-09-13T09:17:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8149999999999996e-06 2024-09-13T09:17:48.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998375+0000", 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024711767999999999 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998484+0000", 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108257 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998586+0000", 2024-09-13T09:17:48.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101953 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4110 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9833124099999999, 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025120632, 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.002 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4110 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:17:48.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:17:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973678+0000", 2024-09-13T09:17:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0712e-05 2024-09-13T09:17:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973685+0000", 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9029999999999998e-06 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973690+0000", 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.955e-06 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998613+0000", 2024-09-13T09:17:48.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024922546 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998695+0000", 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2066999999999999e-05 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998789+0000", 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3813000000000005e-05 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4111 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9832855719999998, 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025376396999999998, 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4111 2024-09-13T09:17:48.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:17:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:17:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973694+0000", 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973697+0000", 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.384e-06 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973703+0000", 2024-09-13T09:17:48.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9970000000000004e-06 2024-09-13T09:17:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973707+0000", 2024-09-13T09:17:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7169999999999999e-06 2024-09-13T09:17:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998817+0000", 2024-09-13T09:17:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025109446000000001 2024-09-13T09:17:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998898+0000", 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1003999999999999e-05 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999071+0000", 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000173337 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4112 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:17:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.983268748, 2024-09-13T09:17:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025596280999999999, 2024-09-13T09:17:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4112 2024-09-13T09:17:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.012 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:17:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:17:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973711+0000", 2024-09-13T09:17:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973713+0000", 2024-09-13T09:17:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2670000000000001e-06 2024-09-13T09:17:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973719+0000", 2024-09-13T09:17:48.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9379999999999997e-06 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973724+0000", 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6809999999999997e-06 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999115+0000", 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025390511000000001 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999202+0000", 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7787999999999994e-05 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999308+0000", 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105568 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4113 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 430080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.016 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9832516030000003, 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025814090000000001, 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4113 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973728+0000", 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973738+0000", 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4339999999999998e-06 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973745+0000", 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9639999999999997e-06 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973754+0000", 2024-09-13T09:17:48.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4760000000000004e-06 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999360+0000", 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025606035999999999 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999443+0000", 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3393000000000006e-05 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999543+0000", 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9351000000000005e-05 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4114 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9832217480000001, 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026036354000000001, 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.021 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4114 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:17:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973758+0000", 2024-09-13T09:17:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973761+0000", 2024-09-13T09:17:48.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973767+0000", 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6209999999999998e-06 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973772+0000", 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999593+0000", 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025820532 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999691+0000", 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8701000000000002e-05 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999795+0000", 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103467 2024-09-13T09:17:48.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4115 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9832040050000002, 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026260549000000001, 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4115 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973778+0000", 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.091e-06 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973784+0000", 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.268e-06 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973789+0000", 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4599999999999996e-06 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999836+0000", 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026046980000000001 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999932+0000", 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5863e-05 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000037+0000", 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105264 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4116 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.983187617, 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026488986999999999, 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4116 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973792+0000", 2024-09-13T09:17:48.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973794+0000", 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8950000000000001e-06 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973800+0000", 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9780000000000002e-06 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973811+0000", 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1192999999999999e-05 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000081+0000", 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026269942000000001 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000175+0000", 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3819000000000004e-05 2024-09-13T09:17:48.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000282+0000", 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106507 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4117 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9831640730000002, 2024-09-13T09:17:48.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026706133999999999, 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4117 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973818+0000", 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6259999999999999e-06 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973825+0000", 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.737e-06 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973830+0000", 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4399999999999998e-06 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000317+0000", 2024-09-13T09:17:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026486948 2024-09-13T09:17:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000398+0000", 2024-09-13T09:17:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1160000000000002e-05 2024-09-13T09:17:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000522+0000", 2024-09-13T09:17:48.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012466800000000001 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4118 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9831465440000002, 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026891327, 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4118 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:17:48.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973833+0000", 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973836+0000", 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2350000000000002e-06 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973841+0000", 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7230000000000001e-06 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973846+0000", 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1829999999999996e-06 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000563+0000", 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026717797000000001 2024-09-13T09:17:48.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000630+0000", 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6152e-05 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000725+0000", 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5601999999999997e-05 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4119 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.983130606, 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027064775999999999, 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4119 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.045 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973849+0000", 2024-09-13T09:17:48.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973852+0000", 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973859+0000", 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7589999999999997e-06 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973864+0000", 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8119999999999998e-06 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000747+0000", 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026882653999999999 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000816+0000", 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8841e-05 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000915+0000", 2024-09-13T09:17:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8933999999999998e-05 2024-09-13T09:17:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14583 3.1f 3:feba6951:::rbd_data.11853ecd5f5f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:17:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:17:48.049 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.42294992999999997, 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.050883949999999997, 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14583 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:17:48.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:17:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534028+0000", 2024-09-13T09:17:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538304+0000", 2024-09-13T09:17:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00427634 2024-09-13T09:17:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538342+0000", 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7567000000000001e-05 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538357+0000", 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4936999999999999e-05 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538411+0000", 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4166000000000003e-05 2024-09-13T09:17:48.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538539+0000", 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000127966 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.539016+0000", 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477014 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:48.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.547890+0000", 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0088736430000000005 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584825+0000", 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036934966 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584866+0000", 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1514000000000003e-05 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584914+0000", 2024-09-13T09:17:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8167000000000001e-05 2024-09-13T09:17:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.056 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.056 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:48.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:17:48.059 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone6 from 0xeaa3e18 (overlap 0x4f49ed8) to 0x0 2024-09-13T09:17:48.059 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:48.054+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:48.142 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-13T09:17:48.159 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:48.156+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:48.244 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:48.244 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:48.244 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:48.244 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:48.258 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:17:48.302 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:48.300+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:48.360 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:48.356+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:48.418 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:48.419 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:48.419 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:48.419 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:48.419 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:48.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:17:48.462 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:48.458+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:48.508 INFO:teuthology.orchestra.run.smithi032.stdout:1072 writesame 0xb531b23 thru 0xb53d59a data_size 0x88(0xba78 bytes) 2024-09-13T09:17:48.524 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:48.526 INFO:teuthology.orchestra.run.smithi032.stdout:1076 trunc from 0xeaa3e18 to 0xcbacd06 2024-09-13T09:17:48.550 INFO:teuthology.orchestra.run.smithi032.stdout:1078 write 0x4fed510 thru 0x4ffc9c7 (0xf4b8 bytes) 2024-09-13T09:17:48.558 INFO:teuthology.orchestra.run.smithi032.stdout:1079 punch from 0x6d0fd7d to 0x6d12ce5, (0x2f68 bytes) 2024-09-13T09:17:48.559 INFO:teuthology.orchestra.run.smithi032.stdout:1082 read 0x16c81e6 thru 0x16d7a01 (0xf81c bytes) 2024-09-13T09:17:48.559 INFO:teuthology.orchestra.run.smithi032.stdout:1083 read 0x617d61 thru 0x6235fb (0xb89b bytes) 2024-09-13T09:17:48.560 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:48.560 INFO:teuthology.orchestra.run.smithi032.stdout:1086 read 0x1c1a171 thru 0x1c25d8c (0xbc1c bytes) 2024-09-13T09:17:48.561 INFO:teuthology.orchestra.run.smithi032.stdout:1087 write 0x8c77f70 thru 0x8c82850 (0xa8e1 bytes) 2024-09-13T09:17:48.562 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:48.558+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:48.570 INFO:teuthology.orchestra.run.smithi032.stdout:1088 read 0x20f204 thru 0x20facf (0x8cc bytes) 2024-09-13T09:17:48.571 INFO:teuthology.orchestra.run.smithi032.stdout:1089 write 0x695f2fd thru 0x69673a6 (0x80aa bytes) 2024-09-13T09:17:48.583 INFO:teuthology.orchestra.run.smithi032.stdout:1090 write 0x8740d83 thru 0x874bcab (0xaf29 bytes) 2024-09-13T09:17:48.591 INFO:teuthology.orchestra.run.smithi032.stdout:1091 write 0x390ad2f thru 0x390e7f8 (0x3aca bytes) 2024-09-13T09:17:48.599 INFO:teuthology.orchestra.run.smithi032.stdout:1092 read 0x3ccd902 thru 0x3cd8671 (0xad70 bytes) 2024-09-13T09:17:48.600 INFO:teuthology.orchestra.run.smithi032.stdout:1094 write 0x24dc4d8 thru 0x24defec (0x2b15 bytes) 2024-09-13T09:17:48.607 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:48.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5478 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:17:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0660708369999998, 2024-09-13T09:17:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027749532, 2024-09-13T09:17:48.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.609 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5478 2024-09-13T09:17:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:17:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:17:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:17:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539236+0000", 2024-09-13T09:17:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.041e-06 2024-09-13T09:17:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539242+0000", 2024-09-13T09:17:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0650000000000004e-06 2024-09-13T09:17:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539246+0000", 2024-09-13T09:17:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1500000000000001e-06 2024-09-13T09:17:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566770+0000", 2024-09-13T09:17:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027523178999999998 2024-09-13T09:17:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566846+0000", 2024-09-13T09:17:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5941999999999999e-05 2024-09-13T09:17:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566984+0000", 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000138486 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5479 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0660545729999997, 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027937460000000001, 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.616 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5479 2024-09-13T09:17:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:17:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539250+0000", 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539256+0000", 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539263+0000", 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3779999999999998e-06 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539273+0000", 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0457e-05 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567010+0000", 2024-09-13T09:17:48.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027736878 2024-09-13T09:17:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567083+0000", 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3187000000000002e-05 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567188+0000", 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010491499999999999 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5480 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.621 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0660273550000001, 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028132641999999999, 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5480 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539280+0000", 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539297+0000", 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7147999999999999e-05 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539303+0000", 2024-09-13T09:17:48.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9039999999999997e-06 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567230+0000", 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027927250000000001 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567316+0000", 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5970999999999994e-05 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567411+0000", 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4523000000000003e-05 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5481 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0659973589999998, 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028317978000000001, 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5481 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:17:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539307+0000", 2024-09-13T09:17:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539311+0000", 2024-09-13T09:17:48.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.264e-06 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539317+0000", 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539322+0000", 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6949999999999999e-06 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567446+0000", 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028124341000000001 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567527+0000", 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0785000000000007e-05 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567626+0000", 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9103000000000003e-05 2024-09-13T09:17:48.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5482 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0659795340000002, 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028619326, 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5482 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539325+0000", 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539333+0000", 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.055e-06 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539340+0000", 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4980000000000003e-06 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539344+0000", 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5369999999999999e-06 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567655+0000", 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028310906 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567732+0000", 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6829999999999995e-05 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567945+0000", 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000212909 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.635 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5483 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0659562730000003, 2024-09-13T09:17:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028784015999999999, 2024-09-13T09:17:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5483 2024-09-13T09:17:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539351+0000", 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539357+0000", 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0090000000000004e-06 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539361+0000", 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.357e-06 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567974+0000", 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028612385000000001 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568050+0000", 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5820000000000003e-05 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568133+0000", 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3551e-05 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5484 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0659397210000003, 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028949923999999998, 2024-09-13T09:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5484 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:17:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539365+0000", 2024-09-13T09:17:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539367+0000", 2024-09-13T09:17:48.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539374+0000", 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539378+0000", 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568159+0000", 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028780490999999998 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568236+0000", 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7156000000000005e-05 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568316+0000", 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9598999999999995e-05 2024-09-13T09:17:48.644 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5485 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0651198419999997, 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028332593, 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.645 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5485 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540189+0000", 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540198+0000", 2024-09-13T09:17:48.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9099999999999994e-06 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540206+0000", 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568360+0000", 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028154324000000001 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568436+0000", 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5821000000000005e-05 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568518+0000", 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1895000000000001e-05 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.649 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5486 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0650668330000004, 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028479536999999999, 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5486 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540241+0000", 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4290000000000001e-06 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540249+0000", 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9480000000000008e-06 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540255+0000", 2024-09-13T09:17:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6019999999999999e-06 2024-09-13T09:17:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568543+0000", 2024-09-13T09:17:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028288260999999999 2024-09-13T09:17:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568636+0000", 2024-09-13T09:17:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2627999999999995e-05 2024-09-13T09:17:48.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568718+0000", 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2213999999999996e-05 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5487 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0650457480000002, 2024-09-13T09:17:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028653435000000001, 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5487 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:17:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540259+0000", 2024-09-13T09:17:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540262+0000", 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.413e-06 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540268+0000", 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3940000000000001e-06 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540272+0000", 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3989999999999997e-06 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568745+0000", 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028472173999999999 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568822+0000", 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7694000000000005e-05 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568913+0000", 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0779000000000006e-05 2024-09-13T09:17:48.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5488 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 430080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0650283480000002, 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028788326999999999, 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5488 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540279+0000", 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2409999999999998e-06 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540285+0000", 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3180000000000003e-06 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540300+0000", 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4625e-05 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568939+0000", 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028639688 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569016+0000", 2024-09-13T09:17:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6556000000000004e-05 2024-09-13T09:17:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569065+0000", 2024-09-13T09:17:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9246000000000003e-05 2024-09-13T09:17:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5489 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:17:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0650007539999997, 2024-09-13T09:17:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028875332, 2024-09-13T09:17:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.664 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5489 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540304+0000", 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540307+0000", 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4370000000000001e-06 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540313+0000", 2024-09-13T09:17:48.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540318+0000", 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569079+0000", 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028760714 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569128+0000", 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8702999999999998e-05 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569180+0000", 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2105999999999999e-05 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.668 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5490 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0649822589999998, 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029041027000000001, 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5490 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540326+0000", 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9840000000000001e-06 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540332+0000", 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9730000000000002e-06 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540338+0000", 2024-09-13T09:17:48.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3550000000000001e-06 2024-09-13T09:17:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569202+0000", 2024-09-13T09:17:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028863521 2024-09-13T09:17:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569276+0000", 2024-09-13T09:17:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:17:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569364+0000", 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7594000000000002e-05 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5491 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.064962875, 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029221864, 2024-09-13T09:17:48.673 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5491 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540350+0000", 2024-09-13T09:17:48.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4250000000000001e-06 2024-09-13T09:17:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540356+0000", 2024-09-13T09:17:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2960000000000004e-06 2024-09-13T09:17:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540361+0000", 2024-09-13T09:17:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:17:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569388+0000", 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029026744 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569474+0000", 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6519000000000003e-05 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569564+0000", 2024-09-13T09:17:48.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9969999999999994e-05 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5492 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0649395779999997, 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029401203000000001, 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5492 2024-09-13T09:17:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:17:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:17:48.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540365+0000", 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540368+0000", 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540374+0000", 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9349999999999999e-06 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540378+0000", 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.651e-06 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569591+0000", 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029212557 2024-09-13T09:17:48.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569668+0000", 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569767+0000", 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9061000000000006e-05 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5493 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0649227139999997, 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029599424999999999, 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5493 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.683 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540382+0000", 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540384+0000", 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0430000000000002e-06 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540390+0000", 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540395+0000", 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5990000000000004e-06 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569795+0000", 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029400582000000001 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569880+0000", 2024-09-13T09:17:48.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5210999999999994e-05 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569982+0000", 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101584 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5494 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:17:48.687 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0649066290000002, 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029770637999999999, 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5494 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:17:48.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540398+0000", 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540406+0000", 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540413+0000", 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540418+0000", 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8289999999999998e-06 2024-09-13T09:17:48.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570007+0000", 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029588851999999999 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570084+0000", 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7253e-05 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570169+0000", 2024-09-13T09:17:48.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5391999999999998e-05 2024-09-13T09:17:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5495 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:17:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0648683129999998, 2024-09-13T09:17:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029938664, 2024-09-13T09:17:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5495 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:17:48.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:17:48.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540440+0000", 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0319999999999999e-06 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540446+0000", 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8549999999999996e-06 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540451+0000", 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570198+0000", 2024-09-13T09:17:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029746060000000001 2024-09-13T09:17:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570284+0000", 2024-09-13T09:17:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6044999999999994e-05 2024-09-13T09:17:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570376+0000", 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2071000000000007e-05 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5496 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.064849358, 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030121617999999999, 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.697 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5496 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:17:48.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540458+0000", 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540464+0000", 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.156e-06 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540468+0000", 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570404+0000", 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029935632 2024-09-13T09:17:48.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570488+0000", 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3873000000000001e-05 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570578+0000", 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9757000000000001e-05 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5497 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.064832794, 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030305259000000001, 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5497 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.702 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:48.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540472+0000", 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540474+0000", 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0490000000000002e-06 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540480+0000", 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1970000000000001e-06 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540491+0000", 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0964e-05 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570608+0000", 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030116105000000001 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570688+0000", 2024-09-13T09:17:48.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0143999999999997e-05 2024-09-13T09:17:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570778+0000", 2024-09-13T09:17:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0131999999999997e-05 2024-09-13T09:17:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:48.706 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:48.706 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:48.707 INFO:teuthology.orchestra.run.smithi032.stdout:1096 trunc from 0xcbacd06 to 0xc17495b 2024-09-13T09:17:48.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:17:48.709 INFO:teuthology.orchestra.run.smithi032.stdout:1097 write 0x7272509 thru 0x727c934 (0xa42c bytes) 2024-09-13T09:17:48.709 INFO:teuthology.orchestra.run.smithi032.stdout:1098 trunc from 0xc17495b to 0x52c612 2024-09-13T09:17:48.709 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:48.702+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:48.725 INFO:teuthology.orchestra.run.smithi032.stdout:1099 trunc from 0x52c612 to 0x1088287 2024-09-13T09:17:48.741 INFO:teuthology.orchestra.run.smithi032.stdout:1101 read 0x6b9fad thru 0x6c915d (0xf1b1 bytes) 2024-09-13T09:17:48.760 INFO:teuthology.orchestra.run.smithi032.stdout:1102 write 0x45a17a0 thru 0x45b1643 (0xfea4 bytes) 2024-09-13T09:17:48.767 INFO:teuthology.orchestra.run.smithi032.stdout:1104 read 0x362cfdf thru 0x3635a42 (0x8a64 bytes) 2024-09-13T09:17:48.767 INFO:teuthology.orchestra.run.smithi032.stdout:1105 write 0x33ecf6e thru 0x33f5b16 (0x8ba9 bytes) 2024-09-13T09:17:48.783 INFO:teuthology.orchestra.run.smithi032.stdout:1106 trunc from 0x45b1644 to 0x94085ec 2024-09-13T09:17:48.797 INFO:teuthology.orchestra.run.smithi032.stdout:1107 write 0x33c6b4f thru 0x33d3c0a (0xd0bc bytes) 2024-09-13T09:17:48.804 INFO:teuthology.orchestra.run.smithi032.stdout:1108 write 0x455b0ca thru 0x455dbf8 (0x2b2f bytes) 2024-09-13T09:17:48.809 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:48.804+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:48.812 INFO:teuthology.orchestra.run.smithi032.stdout:1109 read 0x5f25c66 thru 0x5f2c673 (0x6a0e bytes) 2024-09-13T09:17:48.835 INFO:teuthology.orchestra.run.smithi032.stdout:1110 write 0xd2f0c45 thru 0xd2f1fa4 (0x1360 bytes) 2024-09-13T09:17:48.842 INFO:teuthology.orchestra.run.smithi032.stdout:1112 write 0x6363084 thru 0x6368da8 (0x5d25 bytes) 2024-09-13T09:17:48.849 INFO:teuthology.orchestra.run.smithi032.stdout:1113 read 0x28a576 thru 0x2923b1 (0x7e3c bytes) 2024-09-13T09:17:48.865 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:48.866 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:48.871 INFO:teuthology.orchestra.run.smithi032.stdout:1114 trunc from 0xd2f1fa5 to 0x3e98cf3 2024-09-13T09:17:48.884 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:17:48.897 INFO:teuthology.orchestra.run.smithi032.stdout:1115 writesame 0x158e404 thru 0x1593284 data_size 0x57(0x4e81 bytes) 2024-09-13T09:17:48.929 INFO:teuthology.orchestra.run.smithi032.stdout:1117 write 0x956b9fc thru 0x956f35f (0x3964 bytes) 2024-09-13T09:17:48.929 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:48.924+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:48.950 INFO:teuthology.orchestra.run.smithi032.stdout:1119 write 0xa69d8cc thru 0xa69e2e5 (0xa1a bytes) 2024-09-13T09:17:48.956 INFO:teuthology.orchestra.run.smithi032.stdout:1120 read 0xa14e246 thru 0xa15a44c (0xc207 bytes) 2024-09-13T09:17:48.957 INFO:teuthology.orchestra.run.smithi032.stdout:1121 write 0x35db436 thru 0x35e5912 (0xa4dd bytes) 2024-09-13T09:17:48.967 INFO:teuthology.orchestra.run.smithi032.stdout:1122 write 0x12589cd thru 0x125a042 (0x1676 bytes) 2024-09-13T09:17:48.974 INFO:teuthology.orchestra.run.smithi032.stdout:1123 read 0x2c02cc5 thru 0x2c06dd1 (0x410d bytes) 2024-09-13T09:17:48.974 INFO:teuthology.orchestra.run.smithi032.stdout:1125 write 0x1cadd21 thru 0x1caeea1 (0x1181 bytes) 2024-09-13T09:17:48.981 INFO:teuthology.orchestra.run.smithi032.stdout:1126 read 0x7e65489 thru 0x7e6da26 (0x859e bytes) 2024-09-13T09:17:48.981 INFO:teuthology.orchestra.run.smithi032.stdout:1131 write 0x8d92658 thru 0x8d9f15c (0xcb05 bytes) 2024-09-13T09:17:48.988 INFO:teuthology.orchestra.run.smithi032.stdout:1132 read 0x9f5e79a thru 0x9f63f7b (0x57e2 bytes) 2024-09-13T09:17:48.988 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:48.988 INFO:teuthology.orchestra.run.smithi032.stdout:1136 punch from 0x8980f3a to 0x8986417, (0x54dd bytes) 2024-09-13T09:17:49.006 INFO:teuthology.orchestra.run.smithi032.stdout:1137 write 0xcf13084 thru 0xcf1a02e (0x6fab bytes) 2024-09-13T09:17:49.011 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:49.008+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:49.029 INFO:teuthology.orchestra.run.smithi032.stdout:1140 writesame 0xe46adf7 thru 0xe46c8e2 data_size 0x2(0x1aec bytes) 2024-09-13T09:17:49.029 INFO:teuthology.orchestra.run.smithi032.stdout:1143 read 0xdae00cf thru 0xdaee298 (0xe1ca bytes) 2024-09-13T09:17:49.029 INFO:teuthology.orchestra.run.smithi032.stdout:1144 write 0x2bd2654 thru 0x2be088d (0xe23a bytes) 2024-09-13T09:17:49.031 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:49.031 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:49.031 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:49.031 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:49.031 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:49.038 INFO:teuthology.orchestra.run.smithi032.stdout:1147 read 0xcb9a4a8 thru 0xcb9cb5d (0x26b6 bytes) 2024-09-13T09:17:49.038 INFO:teuthology.orchestra.run.smithi032.stdout:1148 read 0xbb8be07 thru 0xbb97c71 (0xbe6b bytes) 2024-09-13T09:17:49.038 INFO:teuthology.orchestra.run.smithi032.stdout:1149 writesame 0x745d1e9 thru 0x746531d data_size 0x1f(0x8135 bytes) 2024-09-13T09:17:49.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:17:49.047 INFO:teuthology.orchestra.run.smithi032.stdout:1150 read 0x5bea975 thru 0x5bef206 (0x4892 bytes) 2024-09-13T09:17:49.047 INFO:teuthology.orchestra.run.smithi032.stdout:1153 writesame 0xce1ea4d thru 0xce2d727 data_size 0x5(0xecdb bytes) 2024-09-13T09:17:49.064 INFO:teuthology.orchestra.run.smithi032.stdout:1155 punch from 0x8cbca6e to 0x8cc817b, (0xb70d bytes) 2024-09-13T09:17:49.064 INFO:teuthology.orchestra.run.smithi032.stdout:1156 write 0x97e6868 thru 0x97ec2af (0x5a48 bytes) 2024-09-13T09:17:49.071 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:49.071 INFO:teuthology.orchestra.run.smithi032.stdout:1158 punch from 0x2009dee to 0x2012585, (0x8797 bytes) 2024-09-13T09:17:49.071 INFO:teuthology.orchestra.run.smithi032.stdout:1159 read 0x2d7f0ff thru 0x2d81863 (0x2765 bytes) 2024-09-13T09:17:49.071 INFO:teuthology.orchestra.run.smithi032.stdout:1160 writesame 0x806d7ad thru 0x806f20d data_size 0x1(0x1a61 bytes) 2024-09-13T09:17:49.080 INFO:teuthology.orchestra.run.smithi032.stdout:1161 read 0x6836057 thru 0x683b824 (0x57ce bytes) 2024-09-13T09:17:49.080 INFO:teuthology.orchestra.run.smithi032.stdout:1163 punch from 0x78ed77d to 0x78f23d1, (0x4c54 bytes) 2024-09-13T09:17:49.080 INFO:teuthology.orchestra.run.smithi032.stdout:1164 punch from 0xb96ea8c to 0xb97682d, (0x7da1 bytes) 2024-09-13T09:17:49.080 INFO:teuthology.orchestra.run.smithi032.stdout:1166 write 0x18697ee thru 0x1875bd6 (0xc3e9 bytes) 2024-09-13T09:17:49.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:17:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:1167 writesame 0xc5f605 thru 0xc6685d data_size 0x1(0x7259 bytes) 2024-09-13T09:17:49.112 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:49.106+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:49.133 INFO:teuthology.orchestra.run.smithi032.stdout:1168 read 0x5d7f815 thru 0x5d8d5cc (0xddb8 bytes) 2024-09-13T09:17:49.133 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:49.133 INFO:teuthology.orchestra.run.smithi032.stdout:1170 read 0x179ebc7 thru 0x17a9642 (0xaa7c bytes) 2024-09-13T09:17:49.133 INFO:teuthology.orchestra.run.smithi032.stdout:1171 read 0x3f5c5a0 thru 0x3f6bdc7 (0xf828 bytes) 2024-09-13T09:17:49.133 INFO:teuthology.orchestra.run.smithi032.stdout:1172 punch from 0x10feba8 to 0x110d746, (0xeb9e bytes) 2024-09-13T09:17:49.134 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:49.134 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:49.134 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:49.134 INFO:teuthology.orchestra.run.smithi032.stdout:1177 write 0x919c24e thru 0x91a025b (0x400e bytes) 2024-09-13T09:17:49.138 INFO:teuthology.orchestra.run.smithi032.stdout:1178 write 0x899b0cc thru 0x89a49c2 (0x98f7 bytes) 2024-09-13T09:17:49.148 INFO:teuthology.orchestra.run.smithi032.stdout:1179 write 0xd3191d6 thru 0xd322881 (0x96ac bytes) 2024-09-13T09:17:49.157 INFO:teuthology.orchestra.run.smithi032.stdout:1180 punch from 0x3ec3fb3 to 0x3ecb60c, (0x7659 bytes) 2024-09-13T09:17:49.157 INFO:teuthology.orchestra.run.smithi032.stdout:1181 write 0x7c4ef77 thru 0x7c5ca54 (0xdade bytes) 2024-09-13T09:17:49.164 INFO:teuthology.orchestra.run.smithi032.stdout:1182 punch from 0x6c272a6 to 0x6c2de62, (0x6bbc bytes) 2024-09-13T09:17:49.164 INFO:teuthology.orchestra.run.smithi032.stdout:1183 writesame 0x4ab558e thru 0x4abee96 data_size 0x9(0x9909 bytes) 2024-09-13T09:17:49.173 INFO:teuthology.orchestra.run.smithi032.stdout:1184 write 0x3076a3e thru 0x307d724 (0x6ce7 bytes) 2024-09-13T09:17:49.174 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:1034 3.9 e28/22 3:94e462d4:::rbd_object_map.1185192b47c4:head v 28'44, mlcod=28'42)", 2024-09-13T09:17:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.296174+0000", 2024-09-13T09:17:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.876518749000001, 2024-09-13T09:17:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021301900000000001, 2024-09-13T09:17:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.296174+0000", 2024-09-13T09:17:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.296174+0000", 2024-09-13T09:17:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.296173+0000", 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.296186+0000", 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3647e-05 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.296215+0000", 2024-09-13T09:17:49.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8416e-05 2024-09-13T09:17:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.296228+0000", 2024-09-13T09:17:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2883e-05 2024-09-13T09:17:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.296331+0000", 2024-09-13T09:17:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010332600000000001 2024-09-13T09:17:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.296394+0000", 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2526000000000004e-05 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.298212+0000", 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001818336 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.298305+0000", 2024-09-13T09:17:49.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2832999999999997e-05 2024-09-13T09:17:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:1041 3.9 e28/22 3:94e462d4:::rbd_object_map.1185192b47c4:head v 28'47, mlcod=28'45)", 2024-09-13T09:17:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.324863+0000", 2024-09-13T09:17:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.847830734, 2024-09-13T09:17:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021172640000000002, 2024-09-13T09:17:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.179 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.324863+0000", 2024-09-13T09:17:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.324863+0000", 2024-09-13T09:17:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.324859+0000", 2024-09-13T09:17:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.324883+0000", 2024-09-13T09:17:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3547e-05 2024-09-13T09:17:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.324925+0000", 2024-09-13T09:17:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2345999999999997e-05 2024-09-13T09:17:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.324943+0000", 2024-09-13T09:17:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7853999999999999e-05 2024-09-13T09:17:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.325073+0000", 2024-09-13T09:17:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012963199999999999 2024-09-13T09:17:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.325146+0000", 2024-09-13T09:17:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2946000000000003e-05 2024-09-13T09:17:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.326893+0000", 2024-09-13T09:17:49.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001747813 2024-09-13T09:17:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.326980+0000", 2024-09-13T09:17:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6403000000000006e-05 2024-09-13T09:17:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:1063 3.9 e28/22 3:94e462d4:::rbd_object_map.1185192b47c4:head v 28'50, mlcod=28'48)", 2024-09-13T09:17:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.362965+0000", 2024-09-13T09:17:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.809727911, 2024-09-13T09:17:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021073799999999998, 2024-09-13T09:17:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.362965+0000", 2024-09-13T09:17:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.362965+0000", 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.362964+0000", 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.362977+0000", 2024-09-13T09:17:49.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3386e-05 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.363005+0000", 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7926999999999999e-05 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.363018+0000", 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2553e-05 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.363128+0000", 2024-09-13T09:17:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110182 2024-09-13T09:17:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.363185+0000", 2024-09-13T09:17:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7556999999999998e-05 2024-09-13T09:17:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.364980+0000", 2024-09-13T09:17:49.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017949929999999999 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.365073+0000", 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2472000000000005e-05 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:1101 3.9 e28/22 3:94e462d4:::rbd_object_map.1185192b47c4:head v 28'57, mlcod=28'55)", 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.449082+0000", 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.723611419000001, 2024-09-13T09:17:49.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021368759999999998, 2024-09-13T09:17:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.449082+0000", 2024-09-13T09:17:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.449082+0000", 2024-09-13T09:17:49.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.449080+0000", 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.449094+0000", 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3943e-05 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.449123+0000", 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8659000000000001e-05 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.449138+0000", 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4966999999999999e-05 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.449248+0000", 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011069900000000001 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.449305+0000", 2024-09-13T09:17:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6796999999999998e-05 2024-09-13T09:17:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.451124+0000", 2024-09-13T09:17:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018183660000000001 2024-09-13T09:17:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.451219+0000", 2024-09-13T09:17:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5174999999999996e-05 2024-09-13T09:17:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:1145 3.0 e28/22 3:0731d9f1:::rbd_data.1185192b47c4.0000000000000032:head v 28'6, mlcod=28'3)", 2024-09-13T09:17:49.194 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.551022+0000", 2024-09-13T09:17:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.621671258999999, 2024-09-13T09:17:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0023143310000000002, 2024-09-13T09:17:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551022+0000", 2024-09-13T09:17:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551022+0000", 2024-09-13T09:17:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551020+0000", 2024-09-13T09:17:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551037+0000", 2024-09-13T09:17:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7156e-05 2024-09-13T09:17:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551081+0000", 2024-09-13T09:17:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3866999999999999e-05 2024-09-13T09:17:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551099+0000", 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8213e-05 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551198+0000", 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8850000000000004e-05 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.551247+0000", 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8813999999999997e-05 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.553257+0000", 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020100679999999998 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.553336+0000", 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9233000000000005e-05 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:1148 3.9 e28/22 3:94e462d4:::rbd_object_map.1185192b47c4:head v 28'66, mlcod=28'64)", 2024-09-13T09:17:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.558011+0000", 2024-09-13T09:17:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.614682433, 2024-09-13T09:17:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002373188, 2024-09-13T09:17:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.558011+0000", 2024-09-13T09:17:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.558011+0000", 2024-09-13T09:17:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.558009+0000", 2024-09-13T09:17:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.558043+0000", 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3782999999999998e-05 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.558070+0000", 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7083000000000001e-05 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.558083+0000", 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3263e-05 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.558199+0000", 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115616 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.558252+0000", 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2782999999999999e-05 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.560291+0000", 2024-09-13T09:17:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020396799999999999 2024-09-13T09:17:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.560384+0000", 2024-09-13T09:17:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2520000000000002e-05 2024-09-13T09:17:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.204 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:1151 3.9 e28/22 3:94e462d4:::rbd_object_map.1185192b47c4:head v 28'68, mlcod=28'64)", 2024-09-13T09:17:49.205 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.560348+0000", 2024-09-13T09:17:49.205 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.612345735, 2024-09-13T09:17:49.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021089540000000001, 2024-09-13T09:17:49.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.205 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.560348+0000", 2024-09-13T09:17:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.560348+0000", 2024-09-13T09:17:49.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.560346+0000", 2024-09-13T09:17:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.560364+0000", 2024-09-13T09:17:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8516e-05 2024-09-13T09:17:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.560399+0000", 2024-09-13T09:17:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4876000000000003e-05 2024-09-13T09:17:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.560418+0000", 2024-09-13T09:17:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9040000000000001e-05 2024-09-13T09:17:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.560487+0000", 2024-09-13T09:17:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8953000000000002e-05 2024-09-13T09:17:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.560516+0000", 2024-09-13T09:17:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8500000000000002e-05 2024-09-13T09:17:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562357+0000", 2024-09-13T09:17:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001840857 2024-09-13T09:17:49.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562456+0000", 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9908000000000006e-05 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:1153 3.9 e28/22 3:94e462d4:::rbd_object_map.1185192b47c4:head v 28'69, mlcod=28'64)", 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.561197+0000", 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.611496172000001, 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0023424919999999998, 2024-09-13T09:17:49.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.561197+0000", 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.561197+0000", 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.561196+0000", 2024-09-13T09:17:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.561207+0000", 2024-09-13T09:17:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.113e-05 2024-09-13T09:17:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.561226+0000", 2024-09-13T09:17:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9389999999999999e-05 2024-09-13T09:17:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.561254+0000", 2024-09-13T09:17:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8070000000000001e-05 2024-09-13T09:17:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.561347+0000", 2024-09-13T09:17:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3292000000000001e-05 2024-09-13T09:17:49.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.561403+0000", 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5229999999999998e-05 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.563455+0000", 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020521839999999999 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.563540+0000", 2024-09-13T09:17:49.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4678999999999993e-05 2024-09-13T09:17:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:1154 3.9 e28/22 3:94e462d4:::rbd_object_map.1185192b47c4:head v 28'70, mlcod=28'66)", 2024-09-13T09:17:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:33.562461+0000", 2024-09-13T09:17:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.610232062, 2024-09-13T09:17:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021087559999999998, 2024-09-13T09:17:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562461+0000", 2024-09-13T09:17:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562461+0000", 2024-09-13T09:17:49.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562459+0000", 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562476+0000", 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6197000000000001e-05 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562513+0000", 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6952999999999997e-05 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562533+0000", 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0097000000000001e-05 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562637+0000", 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010444499999999999 2024-09-13T09:17:49.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.562687+0000", 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9645000000000003e-05 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.564465+0000", 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017784809999999999 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:33.564570+0000", 2024-09-13T09:17:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000104731 2024-09-13T09:17:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:1523 3.9 e29/22 3:94e462d4:::rbd_object_map.1185192b47c4:head v 29'134, mlcod=29'132)", 2024-09-13T09:17:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:34.543325+0000", 2024-09-13T09:17:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.629368175, 2024-09-13T09:17:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021112610000000001, 2024-09-13T09:17:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.220 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.543325+0000", 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.543325+0000", 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.543323+0000", 2024-09-13T09:17:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.543341+0000", 2024-09-13T09:17:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8346e-05 2024-09-13T09:17:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.543372+0000", 2024-09-13T09:17:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0487000000000001e-05 2024-09-13T09:17:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.543385+0000", 2024-09-13T09:17:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3912e-05 2024-09-13T09:17:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.543514+0000", 2024-09-13T09:17:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012840700000000001 2024-09-13T09:17:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.543577+0000", 2024-09-13T09:17:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3582000000000002e-05 2024-09-13T09:17:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.545355+0000", 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017774640000000001 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.545436+0000", 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1435999999999998e-05 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:1545 3.9 e29/22 3:94e462d4:::rbd_object_map.1185192b47c4:head v 29'138, mlcod=29'136)", 2024-09-13T09:17:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:34.600334+0000", 2024-09-13T09:17:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.572359464, 2024-09-13T09:17:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0026055739999999998, 2024-09-13T09:17:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600334+0000", 2024-09-13T09:17:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600334+0000", 2024-09-13T09:17:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600332+0000", 2024-09-13T09:17:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600350+0000", 2024-09-13T09:17:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8450000000000001e-05 2024-09-13T09:17:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600385+0000", 2024-09-13T09:17:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4839000000000003e-05 2024-09-13T09:17:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600404+0000", 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8876999999999999e-05 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600499+0000", 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5008999999999998e-05 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600541+0000", 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2015999999999999e-05 2024-09-13T09:17:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.602891+0000", 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0023506579999999998 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.602939+0000", 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7935000000000001e-05 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:2893 3.9 e31/22 3:9571f390:::rbd_data.118591b29fad.00000000000000a0:head v 31'169, mlcod=31'166)", 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:38.299390+0000", 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.873302913, 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025344059999999999, 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.299390+0000", 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.299390+0000", 2024-09-13T09:17:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.299390+0000", 2024-09-13T09:17:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.299399+0000", 2024-09-13T09:17:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7510000000000003e-06 2024-09-13T09:17:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.299417+0000", 2024-09-13T09:17:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7220000000000001e-05 2024-09-13T09:17:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.299428+0000", 2024-09-13T09:17:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1154e-05 2024-09-13T09:17:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.300006+0000", 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00057788100000000003 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.300067+0000", 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1828000000000005e-05 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.301872+0000", 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001804784 2024-09-13T09:17:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.301925+0000", 2024-09-13T09:17:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2541999999999999e-05 2024-09-13T09:17:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3022 3.9 e31/22 3:91681905:::rbd_data.118591b29fad.0000000000000209:head v 31'171, mlcod=31'169)", 2024-09-13T09:17:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:17:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.704453503, 2024-09-13T09:17:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002956209, 2024-09-13T09:17:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:17:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:17:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468237+0000", 2024-09-13T09:17:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468300+0000", 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3202999999999998e-05 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468353+0000", 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2735999999999998e-05 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468374+0000", 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0930000000000001e-05 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468568+0000", 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000194262 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468627+0000", 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8743000000000003e-05 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.471132+0000", 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025055540000000001 2024-09-13T09:17:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.471196+0000", 2024-09-13T09:17:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3590999999999995e-05 2024-09-13T09:17:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3021 3.1f e31/22 3:f8e79a7b:::rbd_data.118591b29fad.0000000000000208:head v 31'30, mlcod=31'28)", 2024-09-13T09:17:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:38.468407+0000", 2024-09-13T09:17:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.704286004, 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002212912, 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468407+0000", 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468407+0000", 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468406+0000", 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468422+0000", 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6699999999999999e-05 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468449+0000", 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6886999999999999e-05 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468466+0000", 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7073000000000001e-05 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468569+0000", 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102472 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468687+0000", 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000118479 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.470534+0000", 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001847097 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.470620+0000", 2024-09-13T09:17:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5767999999999996e-05 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3310 3.9 e35/22 3:9150c3a6:::rbd_data.118591b29fad.0000000000000070:head v 35'173, mlcod=31'171)", 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.526879+0000", 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.6458143419999995, 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021662729999999998, 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.526879+0000", 2024-09-13T09:17:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.526879+0000", 2024-09-13T09:17:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.526877+0000", 2024-09-13T09:17:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.526897+0000", 2024-09-13T09:17:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9729999999999999e-05 2024-09-13T09:17:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.526939+0000", 2024-09-13T09:17:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:17:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.526958+0000", 2024-09-13T09:17:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9505999999999999e-05 2024-09-13T09:17:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527066+0000", 2024-09-13T09:17:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010747900000000001 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527130+0000", 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4243000000000001e-05 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528957+0000", 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018272620000000001 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.529045+0000", 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8113000000000002e-05 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "pg_update_log_missing(3.4 epoch 35/22 rep_tid 113 entries 35'31 (0'0) error 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head by client.4485.0:3311 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 31'29)", 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.6447729899999999, 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033204409999999999, 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:17:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:17:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527918+0000", 2024-09-13T09:17:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527943+0000", 2024-09-13T09:17:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4752999999999998e-05 2024-09-13T09:17:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527988+0000", 2024-09-13T09:17:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4693000000000001e-05 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528029+0000", 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1155999999999997e-05 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528194+0000", 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164792 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.531241+0000", 2024-09-13T09:17:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003046867 2024-09-13T09:17:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "pg_update_log_missing(3.4 epoch 35/22 rep_tid 116 entries 35'33 (0'0) error 3:24ec887b:::rbd_data.118591b29fad.000000000000007c:head by client.4485.0:3321 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 35'30)", 2024-09-13T09:17:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.534125+0000", 2024-09-13T09:17:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.6385678009999998, 2024-09-13T09:17:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027308850000000002, 2024-09-13T09:17:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:17:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534125+0000", 2024-09-13T09:17:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534125+0000", 2024-09-13T09:17:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534124+0000", 2024-09-13T09:17:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534148+0000", 2024-09-13T09:17:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4233e-05 2024-09-13T09:17:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534236+0000", 2024-09-13T09:17:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7832999999999997e-05 2024-09-13T09:17:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534272+0000", 2024-09-13T09:17:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6195999999999997e-05 2024-09-13T09:17:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534431+0000", 2024-09-13T09:17:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000159362 2024-09-13T09:17:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536856+0000", 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024251210000000001 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3323 3.4 e35/22 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head v 35'34, mlcod=35'31)", 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.6368980640000004, 2024-09-13T09:17:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037621640000000001, 2024-09-13T09:17:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:17:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:17:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535793+0000", 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535811+0000", 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8170000000000001e-05 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535840+0000", 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9105999999999999e-05 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535877+0000", 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7017000000000002e-05 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536001+0000", 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012342899999999999 2024-09-13T09:17:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536073+0000", 2024-09-13T09:17:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2123e-05 2024-09-13T09:17:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539433+0000", 2024-09-13T09:17:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033596630000000001 2024-09-13T09:17:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539557+0000", 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012470600000000001 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3358 3.4 e35/22 3:26bebe25:::rbd_data.118591b29fad.0000000000000095:head v 35'37, mlcod=35'34)", 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.553104+0000", 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.6195887779999998, 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024175500000000001, 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553104+0000", 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553104+0000", 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553102+0000", 2024-09-13T09:17:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553124+0000", 2024-09-13T09:17:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1977e-05 2024-09-13T09:17:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553161+0000", 2024-09-13T09:17:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7171e-05 2024-09-13T09:17:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553181+0000", 2024-09-13T09:17:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.916e-05 2024-09-13T09:17:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553307+0000", 2024-09-13T09:17:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012668600000000001 2024-09-13T09:17:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553356+0000", 2024-09-13T09:17:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000001e-05 2024-09-13T09:17:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.555437+0000", 2024-09-13T09:17:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020812780000000002 2024-09-13T09:17:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.555522+0000", 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5154000000000005e-05 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3364 3.4 e35/22 3:2748ed7f:::rbd_data.118591b29fad.0000000000000215:head v 35'39, mlcod=35'35)", 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.560115+0000", 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.6125785749999997, 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0026426610000000001, 2024-09-13T09:17:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560115+0000", 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560115+0000", 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560113+0000", 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560130+0000", 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7163000000000002e-05 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560164+0000", 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4196999999999998e-05 2024-09-13T09:17:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560186+0000", 2024-09-13T09:17:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1908000000000001e-05 2024-09-13T09:17:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560346+0000", 2024-09-13T09:17:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015956400000000001 2024-09-13T09:17:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560392+0000", 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6113999999999999e-05 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.562667+0000", 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002274501 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.562757+0000", 2024-09-13T09:17:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0802000000000002e-05 2024-09-13T09:17:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:49.276 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:49.277 INFO:teuthology.orchestra.run.smithi032.stdout:1185 trunc from 0xe46c8e3 to 0x8b9d602 2024-09-13T09:17:49.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:17:49.279 INFO:teuthology.orchestra.run.smithi032.stdout:1187 write 0x74a222 thru 0x7563c4 (0xc1a3 bytes) 2024-09-13T09:17:49.279 INFO:teuthology.orchestra.run.smithi032.stdout:1188 trunc from 0x8b9d602 to 0x413ca64 2024-09-13T09:17:49.279 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:49.280 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:49.281 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:49.281 INFO:teuthology.orchestra.run.smithi032.stdout:1189 writesame 0xc5b8e3f thru 0xc5c88a6 data_size 0xc(0xfa68 bytes) 2024-09-13T09:17:49.282 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:49.274+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:49.284 INFO:teuthology.orchestra.run.smithi032.stdout:1190 trunc from 0xc5c88a7 to 0x6e5d586 2024-09-13T09:17:49.289 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:49.309 INFO:teuthology.orchestra.run.smithi032.stdout:1192 write 0x7123a74 thru 0x7128c03 (0x5190 bytes) 2024-09-13T09:17:49.310 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:49.315 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xec4f682 2024-09-13T09:17:49.326 INFO:teuthology.orchestra.run.smithi032.stdout:1193 trunc from 0x7128c04 to 0xec4f682 2024-09-13T09:17:49.338 INFO:teuthology.orchestra.run.smithi032.stdout:1194 punch from 0xd6daa15 to 0xd6e01c1, (0x57ac bytes) 2024-09-13T09:17:49.338 INFO:teuthology.orchestra.run.smithi032.stdout:1195 write 0xb69a048 thru 0xb69a09d (0x56 bytes) 2024-09-13T09:17:49.345 INFO:teuthology.orchestra.run.smithi032.stdout:1196 writesame 0x1076be4 thru 0x10780c4 data_size 0x1(0x14e1 bytes) 2024-09-13T09:17:49.346 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:49.347 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:49.357 INFO:teuthology.orchestra.run.smithi032.stdout:1197 writesame 0xa0912 thru 0xaac37 data_size 0x3(0xa326 bytes) 2024-09-13T09:17:49.368 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:49.369 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:49.378 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:49.374+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:49.387 INFO:teuthology.orchestra.run.smithi032.stdout:1198 trunc from 0xec4f682 to 0x16a255c 2024-09-13T09:17:49.406 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:49.407 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:49.412 INFO:teuthology.orchestra.run.smithi032.stdout:1199 read 0x150c3f6 thru 0x150fd05 (0x3910 bytes) 2024-09-13T09:17:49.428 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:49.428 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:49.435 INFO:teuthology.orchestra.run.smithi032.stdout:1200 write 0x8482ecf thru 0x8483983 (0xab5 bytes) 2024-09-13T09:17:49.442 INFO:teuthology.orchestra.run.smithi032.stdout:1201 read 0x1f976a7 thru 0x1fa2eaa (0xb804 bytes) 2024-09-13T09:17:49.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:17:49.458 INFO:teuthology.orchestra.run.smithi032.stdout:1202 writesame 0x98d6c09 thru 0x98d88db data_size 0x1(0x1cd3 bytes) 2024-09-13T09:17:49.458 INFO:teuthology.orchestra.run.smithi032.stdout:1203 read 0x22d28cf thru 0x22d4f80 (0x26b2 bytes) 2024-09-13T09:17:49.458 INFO:teuthology.orchestra.run.smithi032.stdout:1205 punch from 0x76fc283 to 0x770bcb6, (0xfa33 bytes) 2024-09-13T09:17:49.458 INFO:teuthology.orchestra.run.smithi032.stdout:1206 read 0x617004c thru 0x617fb9e (0xfb53 bytes) 2024-09-13T09:17:49.461 INFO:teuthology.orchestra.run.smithi032.stdout:1207 trunc from 0x98d88dc to 0x80a8066 2024-09-13T09:17:49.479 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:49.476+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:49.482 INFO:teuthology.orchestra.run.smithi032.stdout:1209 read 0x2f4ab0d thru 0x2f4ed72 (0x4266 bytes) 2024-09-13T09:17:49.502 INFO:teuthology.orchestra.run.smithi032.stdout:1210 write 0xaf4f286 thru 0xaf56402 (0x717d bytes) 2024-09-13T09:17:49.508 INFO:teuthology.orchestra.run.smithi032.stdout:1211 read 0xae8422d thru 0xae900f0 (0xbec4 bytes) 2024-09-13T09:17:49.508 INFO:teuthology.orchestra.run.smithi032.stdout:1212 punch from 0x4efd9a8 to 0x4f0d098, (0xf6f0 bytes) 2024-09-13T09:17:49.508 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:49.508 INFO:teuthology.orchestra.run.smithi032.stdout:1214 write 0x7020a3f thru 0x7025bfb (0x51bd bytes) 2024-09-13T09:17:49.514 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:49.529 INFO:teuthology.orchestra.run.smithi032.stdout:1216 trunc from 0xaf56403 to 0x2b92121 2024-09-13T09:17:49.554 INFO:teuthology.orchestra.run.smithi032.stdout:1217 writesame 0x183f064 thru 0x184a8df data_size 0x2(0xb87c bytes) 2024-09-13T09:17:49.580 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:49.576+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:49.593 INFO:teuthology.orchestra.run.smithi032.stdout:1218 read 0x23b08d9 thru 0x23b7cb8 (0x73e0 bytes) 2024-09-13T09:17:49.598 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:49.598 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:49.617 INFO:teuthology.orchestra.run.smithi032.stdout:1219 trunc from 0x2b92121 to 0xd8eaee2 2024-09-13T09:17:49.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:17:49.632 INFO:teuthology.orchestra.run.smithi032.stdout:1220 write 0x72f0e9b thru 0x72f54c6 (0x462c bytes) 2024-09-13T09:17:49.638 INFO:teuthology.orchestra.run.smithi032.stdout:1221 read 0xa343291 thru 0xa34729a (0x400a bytes) 2024-09-13T09:17:49.639 INFO:teuthology.orchestra.run.smithi032.stdout:1222 write 0x3d952c2 thru 0x3d97176 (0x1eb5 bytes) 2024-09-13T09:17:49.645 INFO:teuthology.orchestra.run.smithi032.stdout:1226 punch from 0x70e7f39 to 0x70e8a33, (0xafa bytes) 2024-09-13T09:17:49.645 INFO:teuthology.orchestra.run.smithi032.stdout:1227 write 0xabd4e73 thru 0xabe13a8 (0xc536 bytes) 2024-09-13T09:17:49.655 INFO:teuthology.orchestra.run.smithi032.stdout:1228 write 0xe670e0 thru 0xe734cd (0xc3ee bytes) 2024-09-13T09:17:49.663 INFO:teuthology.orchestra.run.smithi032.stdout:1229 write 0xccf0655 thru 0xccf7cf5 (0x76a1 bytes) 2024-09-13T09:17:49.670 INFO:teuthology.orchestra.run.smithi032.stdout:1230 writesame 0x48fa4e7 thru 0x490134a data_size 0x24(0x6e64 bytes) 2024-09-13T09:17:49.678 INFO:teuthology.orchestra.run.smithi032.stdout:1235 punch from 0xbea9bc to 0xbf578f, (0xadd3 bytes) 2024-09-13T09:17:49.678 INFO:teuthology.orchestra.run.smithi032.stdout:1236 writesame 0x5c04d2e thru 0x5c14084 data_size 0x3(0xf357 bytes) 2024-09-13T09:17:49.681 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:49.676+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:49.711 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:49.711 INFO:teuthology.orchestra.run.smithi032.stdout:1238 punch from 0xbb4d7e0 to 0xbb5c132, (0xe952 bytes) 2024-09-13T09:17:49.711 INFO:teuthology.orchestra.run.smithi032.stdout:1239 write 0x11625e3 thru 0x11712ee (0xed0c bytes) 2024-09-13T09:17:49.718 INFO:teuthology.orchestra.run.smithi032.stdout:1240 read 0x4c4b001 thru 0x4c57b98 (0xcb98 bytes) 2024-09-13T09:17:49.718 INFO:teuthology.orchestra.run.smithi032.stdout:1241 writesame 0x9489bc4 thru 0x948bc92 data_size 0x1(0x20cf bytes) 2024-09-13T09:17:49.734 INFO:teuthology.orchestra.run.smithi032.stdout:1244 writesame 0x1d7e15f thru 0x1d86f50 data_size 0x1(0x8df2 bytes) 2024-09-13T09:17:49.765 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:49.765 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:49.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:17:49.782 INFO:teuthology.orchestra.run.smithi032.stdout:1245 trunc from 0xd8eaee2 to 0xa86fb30 2024-09-13T09:17:49.803 INFO:teuthology.orchestra.run.smithi032.stdout:1246 punch from 0x43bfaa7 to 0x43c71fd, (0x7756 bytes) 2024-09-13T09:17:49.803 INFO:teuthology.orchestra.run.smithi032.stdout:1247 punch from 0x9866deb to 0x9873d40, (0xcf55 bytes) 2024-09-13T09:17:49.821 INFO:teuthology.orchestra.run.smithi032.stdout:1249 trunc from 0xa86fb30 to 0xa0450a 2024-09-13T09:17:49.826 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:49.822+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:49.840 INFO:teuthology.orchestra.run.smithi032.stdout:1250 read 0x6050a9 thru 0x61003f (0xaf97 bytes) 2024-09-13T09:17:49.841 INFO:teuthology.orchestra.run.smithi032.stdout:1251 punch from 0x8e24e6 to 0x8f1d63, (0xf87d bytes) 2024-09-13T09:17:49.841 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:49.841 INFO:teuthology.orchestra.run.smithi032.stdout:1255 punch from 0x6a8897 to 0x6b75f3, (0xed5c bytes) 2024-09-13T09:17:49.853 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:17:49.854 INFO:teuthology.orchestra.run.smithi032.stdout:1256 write 0x34aa334 thru 0x34b0123 (0x5df0 bytes) 2024-09-13T09:17:49.858 INFO:teuthology.orchestra.run.smithi032.stdout:1258 write 0x22ed022 thru 0x22fb881 (0xe860 bytes) 2024-09-13T09:17:49.864 INFO:teuthology.orchestra.run.smithi032.stdout:1259 read 0x16ac987 thru 0x16ad18a (0x804 bytes) 2024-09-13T09:17:49.864 INFO:teuthology.orchestra.run.smithi032.stdout:1260 read 0x30138ca thru 0x301971b (0x5e52 bytes) 2024-09-13T09:17:49.882 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:49.880+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:49.890 INFO:teuthology.orchestra.run.smithi032.stdout:1262 writesame 0xc50159f thru 0xc508e6d data_size 0x1(0x78cf bytes) 2024-09-13T09:17:49.890 INFO:teuthology.orchestra.run.smithi032.stdout:1263 write 0xb80ac13 thru 0xb816cb0 (0xc09e bytes) 2024-09-13T09:17:49.896 INFO:teuthology.orchestra.run.smithi032.stdout:1264 read 0x269a245 thru 0x26a2d89 (0x8b45 bytes) 2024-09-13T09:17:49.896 INFO:teuthology.orchestra.run.smithi032.stdout:1265 read 0x9e13fa8 thru 0x9e17b3b (0x3b94 bytes) 2024-09-13T09:17:49.896 INFO:teuthology.orchestra.run.smithi032.stdout:1268 writesame 0x6dcf778 thru 0x6dd2028 data_size 0x1(0x28b1 bytes) 2024-09-13T09:17:49.913 INFO:teuthology.orchestra.run.smithi032.stdout:1270 write 0x9088c2d thru 0x908efb6 (0x638a bytes) 2024-09-13T09:17:49.922 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:49.923 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:49.923 INFO:teuthology.orchestra.run.smithi032.stdout:1272 trunc from 0xc508e6e to 0xd81ba66 2024-09-13T09:17:49.936 INFO:teuthology.orchestra.run.smithi032.stdout:1273 writesame 0x82eea99 thru 0x82f9eaa data_size 0xc5(0xb412 bytes) 2024-09-13T09:17:49.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:17:49.941 INFO:teuthology.orchestra.run.smithi032.stdout:1276 write 0xb6d7ef1 thru 0xb6e37eb (0xb8fb bytes) 2024-09-13T09:17:49.948 INFO:teuthology.orchestra.run.smithi032.stdout:1277 read 0xd347456 thru 0xd351e4d (0xa9f8 bytes) 2024-09-13T09:17:49.948 INFO:teuthology.orchestra.run.smithi032.stdout:1278 write 0x8bf9bc7 thru 0x8c08fbe (0xf3f8 bytes) 2024-09-13T09:17:49.985 INFO:teuthology.orchestra.run.smithi032.stdout:1280 trunc from 0xd81ba66 to 0x10e6579 2024-09-13T09:17:49.985 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:49.978+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:50.013 INFO:teuthology.orchestra.run.smithi032.stdout:1282 trunc from 0x10e6579 to 0x95cfdc0 2024-09-13T09:17:50.020 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:50.020 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:50.022 INFO:teuthology.orchestra.run.smithi032.stdout:1285 trunc from 0x95cfdc0 to 0x79aaa83 2024-09-13T09:17:50.025 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:17:50.025 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:17:50.025 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-13T09:17:50.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-13T09:17:50.030 INFO:teuthology.orchestra.run.smithi032.stdout:1286 writesame 0x5a6c833 thru 0x5a732d4 data_size 0x1(0x6aa2 bytes) 2024-09-13T09:17:50.051 INFO:teuthology.orchestra.run.smithi032.stdout:1287 punch from 0x6c2e31c to 0x6c2f9bf, (0x16a3 bytes) 2024-09-13T09:17:50.051 INFO:teuthology.orchestra.run.smithi032.stdout:1288 writesame 0x2cd41b0 thru 0x2ce1ec8 data_size 0x39(0xdd19 bytes) 2024-09-13T09:17:50.068 INFO:teuthology.orchestra.run.smithi032.stdout:1289 write 0xb5289c2 thru 0xb5342fa (0xb939 bytes) 2024-09-13T09:17:50.073 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:50.082 INFO:teuthology.orchestra.run.smithi032.stdout:1291 trunc from 0xb5342fb to 0x51df36e 2024-09-13T09:17:50.083 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:50.080+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:50.095 INFO:teuthology.orchestra.run.smithi032.stdout:1292 read 0x4dd10fd thru 0x4ddd1e7 (0xc0eb bytes) 2024-09-13T09:17:50.095 INFO:teuthology.orchestra.run.smithi032.stdout:1293 read 0x24e395d thru 0x24ecff8 (0x969c bytes) 2024-09-13T09:17:50.107 INFO:teuthology.orchestra.run.smithi032.stdout:1294 trunc from 0x51df36e to 0xd970ec7 2024-09-13T09:17:50.117 INFO:teuthology.orchestra.run.smithi032.stdout:1295 read 0x1fbf09d thru 0x1fcefca (0xff2e bytes) 2024-09-13T09:17:50.117 INFO:teuthology.orchestra.run.smithi032.stdout:1296 read 0xa971a45 thru 0xa97b3bc (0x9978 bytes) 2024-09-13T09:17:50.117 INFO:teuthology.orchestra.run.smithi032.stdout:1297 read 0x9b5912e thru 0x9b64d56 (0xbc29 bytes) 2024-09-13T09:17:50.117 INFO:teuthology.orchestra.run.smithi032.stdout:1298 write 0x48f7086 thru 0x48fec08 (0x7b83 bytes) 2024-09-13T09:17:50.123 INFO:teuthology.orchestra.run.smithi032.stdout:1299 read 0x604bbb7 thru 0x605b58d (0xf9d7 bytes) 2024-09-13T09:17:50.123 INFO:teuthology.orchestra.run.smithi032.stdout:1300 read 0x616a350 thru 0x616bb85 (0x1836 bytes) 2024-09-13T09:17:50.123 INFO:teuthology.orchestra.run.smithi032.stdout:1301 read 0xad5c39e thru 0xad64f20 (0x8b83 bytes) 2024-09-13T09:17:50.123 INFO:teuthology.orchestra.run.smithi032.stdout:1302 write 0x2108802 thru 0x21147a1 (0xbfa0 bytes) 2024-09-13T09:17:50.131 INFO:teuthology.orchestra.run.smithi032.stdout:1303 trunc from 0xd970ec7 to 0xbfbc8d8 2024-09-13T09:17:50.140 INFO:teuthology.orchestra.run.smithi032.stdout:1305 read 0x6a6e82c thru 0x6a78393 (0x9b68 bytes) 2024-09-13T09:17:50.147 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:50.147 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:50.153 INFO:teuthology.orchestra.run.smithi032.stdout:1306 write 0xead8d13 thru 0xeadb70b (0x29f9 bytes) 2024-09-13T09:17:50.160 INFO:teuthology.orchestra.run.smithi032.stdout:1307 write 0xb067dac thru 0xb069e4f (0x20a4 bytes) 2024-09-13T09:17:50.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:17:50.165 INFO:teuthology.orchestra.run.smithi032.stdout:1310 write 0xd4ec7d6 thru 0xd4f6e1a (0xa645 bytes) 2024-09-13T09:17:50.173 INFO:teuthology.orchestra.run.smithi032.stdout:1311 trunc from 0xeadb70c to 0xd86b3d7 2024-09-13T09:17:50.184 INFO:teuthology.orchestra.run.smithi032.stdout:1312 read 0x42a770c thru 0x42aa317 (0x2c0c bytes) 2024-09-13T09:17:50.184 INFO:teuthology.orchestra.run.smithi032.stdout:1314 write 0x42eeae0 thru 0x42f1513 (0x2a34 bytes) 2024-09-13T09:17:50.187 INFO:teuthology.orchestra.run.smithi032.stdout:1315 write 0x6ef4062 thru 0x6efdcd5 (0x9c74 bytes) 2024-09-13T09:17:50.191 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:50.191 INFO:teuthology.orchestra.run.smithi032.stdout:1317 read 0x7684d16 thru 0x7687b45 (0x2e30 bytes) 2024-09-13T09:17:50.194 INFO:teuthology.orchestra.run.smithi032.stdout:1321 trunc from 0xd86b3d7 to 0xb22ab7a 2024-09-13T09:17:50.206 INFO:teuthology.orchestra.run.smithi032.stdout:1322 read 0x96c65db thru 0x96c9c7a (0x36a0 bytes) 2024-09-13T09:17:50.206 INFO:teuthology.orchestra.run.smithi032.stdout:1323 read 0xe5e6cb thru 0xe60135 (0x1a6b bytes) 2024-09-13T09:17:50.206 INFO:teuthology.orchestra.run.smithi032.stdout:1324 read 0x89f57a6 thru 0x8a0448c (0xece7 bytes) 2024-09-13T09:17:50.206 INFO:teuthology.orchestra.run.smithi032.stdout:1326 read 0x797c085 thru 0x797df55 (0x1ed1 bytes) 2024-09-13T09:17:50.207 INFO:teuthology.orchestra.run.smithi032.stdout:1327 write 0x58102f1 thru 0x581c4f3 (0xc203 bytes) 2024-09-13T09:17:50.209 INFO:teuthology.orchestra.run.smithi032.stdout:1328 read 0x9a934b2 thru 0x9aa293a (0xf489 bytes) 2024-09-13T09:17:50.209 INFO:teuthology.orchestra.run.smithi032.stdout:1330 write 0x53c916c thru 0x53cb7d0 (0x2665 bytes) 2024-09-13T09:17:50.210 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:50.206+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:50.213 INFO:teuthology.orchestra.run.smithi032.stdout:1334 read 0x9f83384 thru 0x9f8d6d1 (0xa34e bytes) 2024-09-13T09:17:50.213 INFO:teuthology.orchestra.run.smithi032.stdout:1336 punch from 0x66ebba to 0x673413, (0x4859 bytes) 2024-09-13T09:17:50.213 INFO:teuthology.orchestra.run.smithi032.stdout:1337 read 0x9124523 thru 0x91256fc (0x11da bytes) 2024-09-13T09:17:50.220 INFO:teuthology.orchestra.run.smithi032.stdout:1340 writesame 0xbed46fd thru 0xbedfc67 data_size 0x71(0xb56b bytes) 2024-09-13T09:17:50.220 INFO:teuthology.orchestra.run.smithi032.stdout:1342 write 0xb47ca06 thru 0xb47ec74 (0x226f bytes) 2024-09-13T09:17:50.224 INFO:teuthology.orchestra.run.smithi032.stdout:1343 write 0x376f7dc thru 0x377ad08 (0xb52d bytes) 2024-09-13T09:17:50.227 INFO:teuthology.orchestra.run.smithi032.stdout:1344 read 0x5c37083 thru 0x5c3a1db (0x3159 bytes) 2024-09-13T09:17:50.248 INFO:teuthology.orchestra.run.smithi032.stdout:1345 trunc from 0xbedfc68 to 0x6dad1c 2024-09-13T09:17:50.265 INFO:teuthology.orchestra.run.smithi032.stdout:1346 punch from 0x5a686f to 0x5b5965, (0xf0f6 bytes) 2024-09-13T09:17:50.265 INFO:teuthology.orchestra.run.smithi032.stdout:1348 read 0xe9361 thru 0xec534 (0x31d4 bytes) 2024-09-13T09:17:50.285 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:50.278+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:50.292 INFO:teuthology.orchestra.run.smithi032.stdout:1349 write 0x98ba85f thru 0x98bfbe2 (0x5384 bytes) 2024-09-13T09:17:50.299 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:50.299 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:50.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:17:50.319 INFO:teuthology.orchestra.run.smithi032.stdout:1350 write 0xc63f45d thru 0xc6490fc (0x9ca0 bytes) 2024-09-13T09:17:50.328 INFO:teuthology.orchestra.run.smithi032.stdout:1352 write 0x64b396b thru 0x64ba33b (0x69d1 bytes) 2024-09-13T09:17:50.333 INFO:teuthology.orchestra.run.smithi032.stdout:1353 punch from 0x94c81c2 to 0x94d03f8, (0x8236 bytes) 2024-09-13T09:17:50.333 INFO:teuthology.orchestra.run.smithi032.stdout:1355 punch from 0x4b2b49f to 0x4b30f64, (0x5ac5 bytes) 2024-09-13T09:17:50.333 INFO:teuthology.orchestra.run.smithi032.stdout:1356 read 0x57038f7 thru 0x570c75d (0x8e67 bytes) 2024-09-13T09:17:50.333 INFO:teuthology.orchestra.run.smithi032.stdout:1357 writesame 0xb97deec thru 0xb9881ad data_size 0x2(0xa2c2 bytes) 2024-09-13T09:17:50.351 INFO:teuthology.orchestra.run.smithi032.stdout:1359 write 0x140a2ac thru 0x141314f (0x8ea4 bytes) 2024-09-13T09:17:50.356 INFO:teuthology.orchestra.run.smithi032.stdout:1360 write 0x688ceee thru 0x68930aa (0x61bd bytes) 2024-09-13T09:17:50.368 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-09-13T09:17:50.364+0000 7f1d86e9f640 -1 mon.a@0(leader).osd e40 definitely_dead 0 2024-09-13T09:17:50.371 INFO:teuthology.orchestra.run.smithi032.stdout:1361 write 0xc9edbd9 thru 0xc9f7ca8 (0xa0d0 bytes) 2024-09-13T09:17:50.376 INFO:teuthology.orchestra.run.smithi032.stdout:1362 write 0x902862b thru 0x90380c9 (0xfa9f bytes) 2024-09-13T09:17:50.381 INFO:teuthology.orchestra.run.smithi032.stdout:1363 read 0x9eb1fa4 thru 0x9ebd84f (0xb8ac bytes) 2024-09-13T09:17:50.381 INFO:teuthology.orchestra.run.smithi032.stdout:1364 punch from 0x6cc1561 to 0x6cd1021, (0xfac0 bytes) 2024-09-13T09:17:50.404 INFO:teuthology.orchestra.run.smithi032.stdout:1365 trunc from 0xc9f7ca9 to 0x2c5be84 2024-09-13T09:17:50.406 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:50.402+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:50.431 INFO:teuthology.orchestra.run.smithi032.stdout:1367 trunc from 0x2c5be84 to 0x67cee5b 2024-09-13T09:17:50.441 INFO:teuthology.orchestra.run.smithi032.stdout:1368 read 0x6459f thru 0x65f4d (0x19af bytes) 2024-09-13T09:17:50.442 INFO:teuthology.orchestra.run.smithi032.stdout:1369 read 0xa4b10e thru 0xa4f033 (0x3f26 bytes) 2024-09-13T09:17:50.442 INFO:teuthology.orchestra.run.smithi032.stdout:1370 read 0x4e3337c thru 0x4e3c86d (0x94f2 bytes) 2024-09-13T09:17:50.443 INFO:teuthology.orchestra.run.smithi032.stdout:1371 read 0x490d18f thru 0x491af4f (0xddc1 bytes) 2024-09-13T09:17:50.466 INFO:teuthology.orchestra.run.smithi032.stdout:1372 write 0xb229bc2 thru 0xb22ff88 (0x63c7 bytes) 2024-09-13T09:17:50.474 INFO:teuthology.orchestra.run.smithi032.stdout:1373 write 0x8f64dc6 thru 0x8f6e648 (0x9883 bytes) 2024-09-13T09:17:50.482 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:50.482 INFO:teuthology.orchestra.run.smithi032.stdout:1375 read 0x157d3b5 thru 0x157ee14 (0x1a60 bytes) 2024-09-13T09:17:50.482 INFO:teuthology.orchestra.run.smithi032.stdout:1377 read 0x71a6895 thru 0x71a6c37 (0x3a3 bytes) 2024-09-13T09:17:50.482 INFO:teuthology.orchestra.run.smithi032.stdout:1378 punch from 0x51ec78b to 0x51f5be6, (0x945b bytes) 2024-09-13T09:17:50.482 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:50.486 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:50.482+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:50.488 INFO:teuthology.orchestra.run.smithi032.stdout:1380 trunc from 0xb22ff89 to 0xd509074 2024-09-13T09:17:50.499 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:50.499 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:50.501 INFO:teuthology.orchestra.run.smithi032.stdout:1381 clone 8 order 23 su 4194304 sc 13 2024-09-13T09:17:50.514 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:17:50.602 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:50.598+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:50.684 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:50.684 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:50.687 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:50.682+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:50.698 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:17:50.789 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:50.788+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:50.848 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:17:50.852 INFO:teuthology.orchestra.run.smithi032.stderr:marked out osd.1. 2024-09-13T09:17:50.873 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:50.874 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:50.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:17:50.926 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:50.924+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:50.991 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:50.986+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:51.009 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:51.009 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:51.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:17:51.091 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:51.088+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:51.190 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:51.190 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:51.193 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:51.188+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:51.204 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:17:51.293 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:51.288+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:51.322 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:51.322 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:51.337 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:17:51.394 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:51.396+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:51.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:17:51.464 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:51.465 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:51.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:17:51.495 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:51.492+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:51.590 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:51.591 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:51.596 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:51.590+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:51.598 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:51.598 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:51.607 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:51.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:17:51.623 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:51.641 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:51.641 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:51.661 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:51.662 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:51.682 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:51.701 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:51.701 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:51.702 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:51.698+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:51.723 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:51.780 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:17:51.780 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:17:51.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:17:51.780 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:17:51.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.780 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:17:51.781 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:51.781 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.384309139999999, 2024-09-13T09:17:51.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077478195999999999, 2024-09-13T09:17:51.781 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.781 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:51.781 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.781 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:17:51.781 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:17:51.781 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:17:51.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.781 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:51.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:17:51.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392636+0000", 2024-09-13T09:17:51.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:51.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392651+0000", 2024-09-13T09:17:51.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:17:51.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392674+0000", 2024-09-13T09:17:51.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.264e-05 2024-09-13T09:17:51.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392691+0000", 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7203000000000001e-05 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392780+0000", 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8671999999999994e-05 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:17:51.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392794+0000", 2024-09-13T09:17:51.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-13T09:17:51.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.466664+0000", 2024-09-13T09:17:51.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073869913999999995 2024-09-13T09:17:51.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467139+0000", 2024-09-13T09:17:51.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047488599999999999 2024-09-13T09:17:51.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:17:51.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467664+0000", 2024-09-13T09:17:51.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052530600000000002 2024-09-13T09:17:51.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:17:51.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469125+0000", 2024-09-13T09:17:51.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001460432 2024-09-13T09:17:51.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469847+0000", 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072197799999999996 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470006+0000", 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015955900000000001 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470116+0000", 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109836 2024-09-13T09:17:51.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.788 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.788 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5213 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.788 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538498+0000", 2024-09-13T09:17:51.788 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2384492659999999, 2024-09-13T09:17:51.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049914764, 2024-09-13T09:17:51.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5213 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538498+0000", 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538498+0000", 2024-09-13T09:17:51.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538497+0000", 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538506+0000", 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2129999999999997e-06 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538515+0000", 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538522+0000", 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.007e-06 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588044+0000", 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049521823999999999 2024-09-13T09:17:51.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588264+0000", 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022035199999999999 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588413+0000", 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014863499999999999 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5214 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538528+0000", 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2384193659999996, 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050251938000000003, 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5214 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.793 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538528+0000", 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538528+0000", 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538527+0000", 2024-09-13T09:17:51.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538530+0000", 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2499999999999998e-06 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538539+0000", 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.14e-06 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538562+0000", 2024-09-13T09:17:51.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3107000000000001e-05 2024-09-13T09:17:51.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588494+0000", 2024-09-13T09:17:51.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049932257000000001 2024-09-13T09:17:51.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588615+0000", 2024-09-13T09:17:51.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120932 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588780+0000", 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000164869 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5215 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538572+0000", 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2383754800000002, 2024-09-13T09:17:51.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050803344, 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5215 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538572+0000", 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538572+0000", 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538571+0000", 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538575+0000", 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538585+0000", 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8400000000000007e-06 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538595+0000", 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0346000000000001e-05 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.588863+0000", 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050268238 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.589092+0000", 2024-09-13T09:17:51.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022874500000000001 2024-09-13T09:17:51.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.589375+0000", 2024-09-13T09:17:51.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028307799999999998 2024-09-13T09:17:51.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.801 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5216 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 450560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538638+0000", 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2383089539999999, 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051242401, 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5216 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538638+0000", 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538638+0000", 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538637+0000", 2024-09-13T09:17:51.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538644+0000", 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538680+0000", 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.663e-05 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538697+0000", 2024-09-13T09:17:51.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7033000000000001e-05 2024-09-13T09:17:51.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.589442+0000", 2024-09-13T09:17:51.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050745014999999997 2024-09-13T09:17:51.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.589619+0000", 2024-09-13T09:17:51.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000176338 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.589881+0000", 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000261795 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5217 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538710+0000", 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2382370439999999, 2024-09-13T09:17:51.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051702696999999999, 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5217 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538710+0000", 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538710+0000", 2024-09-13T09:17:51.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538709+0000", 2024-09-13T09:17:51.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:51.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538719+0000", 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4129999999999995e-06 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538738+0000", 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9196000000000001e-05 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538755+0000", 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7413000000000001e-05 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.589952+0000", 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051197222000000001 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590167+0000", 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000214444 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590413+0000", 2024-09-13T09:17:51.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024592199999999998 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5218 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538768+0000", 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2381795479999997, 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052032464, 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5218 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538768+0000", 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538768+0000", 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538767+0000", 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538773+0000", 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1e-06 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538790+0000", 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7266999999999999e-05 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538803+0000", 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3179999999999999e-05 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590503+0000", 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051700044000000001 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590645+0000", 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000141435 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590800+0000", 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015541200000000001 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5219 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538814+0000", 2024-09-13T09:17:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2381330750000004, 2024-09-13T09:17:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052311642999999998, 2024-09-13T09:17:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5219 2024-09-13T09:17:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538814+0000", 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538814+0000", 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538813+0000", 2024-09-13T09:17:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538819+0000", 2024-09-13T09:17:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:17:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538836+0000", 2024-09-13T09:17:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:17:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538849+0000", 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2896e-05 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590838+0000", 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051988922 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.590954+0000", 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116422 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591126+0000", 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017145600000000001 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5220 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538860+0000", 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2380868789999999, 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052651584000000001, 2024-09-13T09:17:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5220 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538860+0000", 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538860+0000", 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538859+0000", 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538865+0000", 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4639999999999997e-06 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538884+0000", 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8963000000000001e-05 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538897+0000", 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3123e-05 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591215+0000", 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052317782 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591355+0000", 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014062600000000001 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591512+0000", 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000156606 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5221 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538908+0000", 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2380390630000004, 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052944665000000002, 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5221 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538908+0000", 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538908+0000", 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538907+0000", 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538913+0000", 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7429999999999999e-06 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538929+0000", 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6609999999999999e-05 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538948+0000", 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8697000000000001e-05 2024-09-13T09:17:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591582+0000", 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052633390000000002 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591709+0000", 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012750899999999999 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591853+0000", 2024-09-13T09:17:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014367899999999999 2024-09-13T09:17:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5222 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 458752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.538961+0000", 2024-09-13T09:17:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2379865590000003, 2024-09-13T09:17:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053306521000000003, 2024-09-13T09:17:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5222 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538961+0000", 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538961+0000", 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538960+0000", 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.538965+0000", 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7230000000000001e-06 2024-09-13T09:17:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539003+0000", 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7323000000000002e-05 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539018+0000", 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5492999999999999e-05 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.591937+0000", 2024-09-13T09:17:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052918924999999999 2024-09-13T09:17:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592115+0000", 2024-09-13T09:17:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000178388 2024-09-13T09:17:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592267+0000", 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015165600000000001 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5223 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539031+0000", 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2379160039999997, 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053594032999999999, 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5223 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539031+0000", 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539031+0000", 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539030+0000", 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539037+0000", 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6359999999999997e-06 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539054+0000", 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7354e-05 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539068+0000", 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3633e-05 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592349+0000", 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053281308999999999 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592484+0000", 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134812 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592625+0000", 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000141345 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5224 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539083+0000", 2024-09-13T09:17:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2378642339999999, 2024-09-13T09:17:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0539636, 2024-09-13T09:17:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5224 2024-09-13T09:17:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539083+0000", 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539083+0000", 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539082+0000", 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539088+0000", 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539106+0000", 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7543000000000001e-05 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539119+0000", 2024-09-13T09:17:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3363000000000001e-05 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592700+0000", 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053580790000000003 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.592907+0000", 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000207161 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593046+0000", 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013960599999999999 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5225 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539130+0000", 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2378172679999997, 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054364970999999998, 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5225 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539130+0000", 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539130+0000", 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539129+0000", 2024-09-13T09:17:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539149+0000", 2024-09-13T09:17:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0095999999999999e-05 2024-09-13T09:17:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539167+0000", 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8284e-05 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539183+0000", 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5645999999999999e-05 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593117+0000", 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053933794 2024-09-13T09:17:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593339+0000", 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022261500000000001 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593495+0000", 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015542900000000001 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5226 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539196+0000", 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2377512309999998, 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054730256999999997, 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5226 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539196+0000", 2024-09-13T09:17:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539196+0000", 2024-09-13T09:17:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539195+0000", 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539201+0000", 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3600000000000001e-06 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539239+0000", 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539259+0000", 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9656999999999998e-05 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593577+0000", 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054317773999999999 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593748+0000", 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017108499999999999 2024-09-13T09:17:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.593926+0000", 2024-09-13T09:17:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017829200000000001 2024-09-13T09:17:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5227 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539286+0000", 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2376614159999999, 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055249030999999997, 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5227 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539286+0000", 2024-09-13T09:17:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539286+0000", 2024-09-13T09:17:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539285+0000", 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539290+0000", 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1000000000000003e-06 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539312+0000", 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2276000000000002e-05 2024-09-13T09:17:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539321+0000", 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9099999999999994e-06 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.594016+0000", 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054694995000000003 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.594252+0000", 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023619799999999999 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.594535+0000", 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028231799999999999 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5228 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539328+0000", 2024-09-13T09:17:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2376186020000004, 2024-09-13T09:17:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055582577000000001, 2024-09-13T09:17:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5228 2024-09-13T09:17:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539328+0000", 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539328+0000", 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539328+0000", 2024-09-13T09:17:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539332+0000", 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539341+0000", 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5629999999999998e-06 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6299999999999998e-06 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.594616+0000", 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055267509999999999 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.594733+0000", 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011629399999999999 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.594911+0000", 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000178567 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5229 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539355+0000", 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2375924659999997, 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.056017574000000001, 2024-09-13T09:17:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5229 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539355+0000", 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539355+0000", 2024-09-13T09:17:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539354+0000", 2024-09-13T09:17:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539358+0000", 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3470000000000001e-06 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539367+0000", 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6230000000000003e-06 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539376+0000", 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7469999999999994e-06 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595014+0000", 2024-09-13T09:17:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055638340000000001 2024-09-13T09:17:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595197+0000", 2024-09-13T09:17:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018309199999999999 2024-09-13T09:17:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595372+0000", 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017503500000000001 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5230 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539384+0000", 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.23756282, 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.056529714000000002, 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5230 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539384+0000", 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539384+0000", 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539384+0000", 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539387+0000", 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2669999999999998e-06 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539395+0000", 2024-09-13T09:17:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2859999999999999e-06 2024-09-13T09:17:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539402+0000", 2024-09-13T09:17:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:17:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595457+0000", 2024-09-13T09:17:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.056054567999999999 2024-09-13T09:17:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595675+0000", 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000218128 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595914+0000", 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000239328 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:5231 3.13 3:cc3395ed:::rbd_data.1185be6cd1eb.0000000000000005:6 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:44.539412+0000", 2024-09-13T09:17:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.2375355729999997, 2024-09-13T09:17:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.057015146000000003, 2024-09-13T09:17:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:17:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:17:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:17:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:17:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 5231 2024-09-13T09:17:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:17:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:17:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539412+0000", 2024-09-13T09:17:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:17:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539412+0000", 2024-09-13T09:17:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:17:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539411+0000", 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539414+0000", 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539423+0000", 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4200000000000007e-06 2024-09-13T09:17:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:17:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.539430+0000", 2024-09-13T09:17:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1330000000000001e-06 2024-09-13T09:17:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:17:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.595979+0000", 2024-09-13T09:17:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.056549006999999998 2024-09-13T09:17:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.596198+0000", 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219292 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.596427+0000", 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000228691 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:17:51.876 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:17:51.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:17:51.879 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:51.876+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:51.979 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:51.976+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:51.988 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone7 from 0xd509074 (overlap 0x52c612) to 0x0 2024-09-13T09:17:52.037 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:52.037 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:52.037 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:52.037 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:52.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:17:52.080 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:52.074+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:52.122 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-13T09:17:52.125 INFO:teuthology.orchestra.run.smithi032.stdout:1383 punch from 0x5294b75 to 0x529d1ef, (0x867a bytes) 2024-09-13T09:17:52.131 INFO:teuthology.orchestra.run.smithi032.stdout:1384 punch from 0x5bc8dea to 0x5bcafea, (0x2200 bytes) 2024-09-13T09:17:52.131 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:52.131 INFO:teuthology.orchestra.run.smithi032.stdout:1386 write 0x5c0506e thru 0x5c133cd (0xe360 bytes) 2024-09-13T09:17:52.137 INFO:teuthology.orchestra.run.smithi032.stdout:1387 write 0x4d8ac26 thru 0x4d944eb (0x98c6 bytes) 2024-09-13T09:17:52.144 INFO:teuthology.orchestra.run.smithi032.stdout:1389 trunc from 0xd509074 to 0xcb8e571 2024-09-13T09:17:52.159 INFO:teuthology.orchestra.run.smithi032.stdout:1390 write 0x955813a thru 0x955e72b (0x65f2 bytes) 2024-09-13T09:17:52.162 INFO:teuthology.orchestra.run.smithi032.stdout:1391 read 0xc7e0de2 thru 0xc7eedb4 (0xdfd3 bytes) 2024-09-13T09:17:52.162 INFO:teuthology.orchestra.run.smithi032.stdout:1392 write 0x3121ab7 thru 0x31270a8 (0x55f2 bytes) 2024-09-13T09:17:52.172 INFO:teuthology.orchestra.run.smithi032.stdout:1394 trunc from 0xcb8e571 to 0xabce0a5 2024-09-13T09:17:52.180 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:52.176+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:52.196 INFO:teuthology.orchestra.run.smithi032.stdout:1396 read 0x69bb7ff thru 0x69c3d08 (0x850a bytes) 2024-09-13T09:17:52.196 INFO:teuthology.orchestra.run.smithi032.stdout:1397 write 0x530052 thru 0x5315fc (0x15ab bytes) 2024-09-13T09:17:52.209 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:52.209 INFO:teuthology.orchestra.run.smithi032.stdout:1399 write 0x32ba93c thru 0x32c2a5a (0x811f bytes) 2024-09-13T09:17:52.222 INFO:teuthology.orchestra.run.smithi032.stdout:1401 trunc from 0xabce0a5 to 0x3f6863d 2024-09-13T09:17:52.235 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:52.236 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:52.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:17:52.259 INFO:teuthology.orchestra.run.smithi032.stdout:1402 write 0xfd95d6 thru 0xfe794c (0xe377 bytes) 2024-09-13T09:17:52.263 INFO:teuthology.orchestra.run.smithi032.stdout:1403 read 0x3f5f6fd thru 0x3f60e87 (0x178b bytes) 2024-09-13T09:17:52.263 INFO:teuthology.orchestra.run.smithi032.stdout:1405 read 0x9138e6 thru 0x917827 (0x3f42 bytes) 2024-09-13T09:17:52.278 INFO:teuthology.orchestra.run.smithi032.stdout:1406 writesame 0x7afbbdf thru 0x7b09721 data_size 0x1(0xdb43 bytes) 2024-09-13T09:17:52.278 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:52.278 INFO:teuthology.orchestra.run.smithi032.stdout:1408 writesame 0x71b2a54 thru 0x71bcda8 data_size 0x1(0xa355 bytes) 2024-09-13T09:17:52.281 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:52.276+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:52.336 INFO:teuthology.orchestra.run.smithi032.stdout:1409 writesame 0x513b06d thru 0x513c812 data_size 0x3(0x17a6 bytes) 2024-09-13T09:17:52.363 INFO:teuthology.orchestra.run.smithi032.stdout:1410 writesame 0xc7d027a thru 0xc7d1974 data_size 0x3(0x16fb bytes) 2024-09-13T09:17:52.363 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:52.363 INFO:teuthology.orchestra.run.smithi032.stdout:1412 read 0x17949e3 thru 0x179c198 (0x77b6 bytes) 2024-09-13T09:17:52.364 INFO:teuthology.orchestra.run.smithi032.stdout:1413 writesame 0x8415722 thru 0x841907e data_size 0x37(0x395d bytes) 2024-09-13T09:17:52.370 INFO:teuthology.orchestra.run.smithi032.stdout:1416 read 0x2436e9 thru 0x24c60e (0x8f26 bytes) 2024-09-13T09:17:52.372 INFO:teuthology.orchestra.run.smithi032.stdout:1417 write 0x1e8c79f thru 0x1e8e7e1 (0x2043 bytes) 2024-09-13T09:17:52.376 INFO:teuthology.orchestra.run.smithi032.stdout:1419 writesame 0xb4c4aa9 thru 0xb4ca64a data_size 0x2(0x5ba2 bytes) 2024-09-13T09:17:52.382 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:52.378+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:52.393 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3:ed699f0f:::rbd_data.118591b29fad.000000000000008c:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:17:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.5296985999999997, 2024-09-13T09:17:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94878089700000001, 2024-09-13T09:17:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:17:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:17:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810654+0000", 2024-09-13T09:17:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94834837599999999 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810755+0000", 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100987 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810828+0000", 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2751999999999997e-05 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4102 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973478+0000", 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4182675769999999, 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023485389999999998, 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4102 2024-09-13T09:17:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973478+0000", 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973478+0000", 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973478+0000", 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973480+0000", 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973486+0000", 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.156e-06 2024-09-13T09:17:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973491+0000", 2024-09-13T09:17:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7790000000000003e-06 2024-09-13T09:17:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.996766+0000", 2024-09-13T09:17:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023274522999999998 2024-09-13T09:17:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.996862+0000", 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6371000000000003e-05 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.996964+0000", 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010140399999999999 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4103 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973495+0000", 2024-09-13T09:17:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4182503559999997, 2024-09-13T09:17:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023681364999999999, 2024-09-13T09:17:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4103 2024-09-13T09:17:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973495+0000", 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973495+0000", 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973495+0000", 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973497+0000", 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3190000000000002e-06 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973503+0000", 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0909999999999998e-06 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973508+0000", 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.996995+0000", 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023486717000000001 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997080+0000", 2024-09-13T09:17:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5500999999999993e-05 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997177+0000", 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6325999999999999e-05 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4104 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:17:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4182208420000002, 2024-09-13T09:17:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023862547000000001, 2024-09-13T09:17:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4104 2024-09-13T09:17:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973524+0000", 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973527+0000", 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1200000000000002e-06 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973535+0000", 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2479999999999999e-06 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973548+0000", 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.365e-05 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997205+0000", 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023657088999999999 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997289+0000", 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3801000000000006e-05 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997387+0000", 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8317000000000001e-05 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4105 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4181923249999997, 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024049968000000001, 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4105 2024-09-13T09:17:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:17:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:17:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:17:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973556+0000", 2024-09-13T09:17:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7420000000000002e-06 2024-09-13T09:17:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973564+0000", 2024-09-13T09:17:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-13T09:17:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973569+0000", 2024-09-13T09:17:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-13T09:17:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997418+0000", 2024-09-13T09:17:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023849052999999999 2024-09-13T09:17:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997504+0000", 2024-09-13T09:17:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6075999999999994e-05 2024-09-13T09:17:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997603+0000", 2024-09-13T09:17:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9455000000000002e-05 2024-09-13T09:17:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4106 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:17:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4181727340000005, 2024-09-13T09:17:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024364336, 2024-09-13T09:17:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4106 2024-09-13T09:17:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:17:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:17:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:17:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973575+0000", 2024-09-13T09:17:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.322e-06 2024-09-13T09:17:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973594+0000", 2024-09-13T09:17:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9031000000000001e-05 2024-09-13T09:17:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973601+0000", 2024-09-13T09:17:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5300000000000002e-06 2024-09-13T09:17:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997648+0000", 2024-09-13T09:17:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024047731999999999 2024-09-13T09:17:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997748+0000", 2024-09-13T09:17:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9807000000000002e-05 2024-09-13T09:17:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997937+0000", 2024-09-13T09:17:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018924299999999999 2024-09-13T09:17:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4107 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:17:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4181402900000002, 2024-09-13T09:17:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542690999999998, 2024-09-13T09:17:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4107 2024-09-13T09:17:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:17:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:17:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:17:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973614+0000", 2024-09-13T09:17:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:17:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973621+0000", 2024-09-13T09:17:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0110000000000003e-06 2024-09-13T09:17:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973626+0000", 2024-09-13T09:17:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0320000000000003e-06 2024-09-13T09:17:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997979+0000", 2024-09-13T09:17:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024353063000000001 2024-09-13T09:17:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998059+0000", 2024-09-13T09:17:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9221000000000006e-05 2024-09-13T09:17:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998148+0000", 2024-09-13T09:17:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9506999999999995e-05 2024-09-13T09:17:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4108 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:17:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4181146820000006, 2024-09-13T09:17:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024717223999999999, 2024-09-13T09:17:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4108 2024-09-13T09:17:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:17:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:17:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:17:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973633+0000", 2024-09-13T09:17:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2089999999999999e-06 2024-09-13T09:17:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973639+0000", 2024-09-13T09:17:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1469999999999998e-06 2024-09-13T09:17:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973644+0000", 2024-09-13T09:17:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5310000000000003e-06 2024-09-13T09:17:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998175+0000", 2024-09-13T09:17:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024531826999999999 2024-09-13T09:17:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998257+0000", 2024-09-13T09:17:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1668999999999993e-05 2024-09-13T09:17:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998348+0000", 2024-09-13T09:17:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1184e-05 2024-09-13T09:17:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4109 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:17:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4180951509999993, 2024-09-13T09:17:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024935052999999999, 2024-09-13T09:17:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4109 2024-09-13T09:17:52.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.442 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:17:52.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:17:52.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973650+0000", 2024-09-13T09:17:52.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973653+0000", 2024-09-13T09:17:52.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.328e-06 2024-09-13T09:17:52.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973659+0000", 2024-09-13T09:17:52.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3269999999999997e-06 2024-09-13T09:17:52.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973664+0000", 2024-09-13T09:17:52.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8149999999999996e-06 2024-09-13T09:17:52.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998375+0000", 2024-09-13T09:17:52.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024711767999999999 2024-09-13T09:17:52.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998484+0000", 2024-09-13T09:17:52.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108257 2024-09-13T09:17:52.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998586+0000", 2024-09-13T09:17:52.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101953 2024-09-13T09:17:52.446 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.447 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.447 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4110 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.447 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:17:52.447 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4180778160000003, 2024-09-13T09:17:52.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025120632, 2024-09-13T09:17:52.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.447 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.448 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.448 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.448 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.448 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4110 2024-09-13T09:17:52.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.448 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:17:52.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:17:52.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:17:52.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973678+0000", 2024-09-13T09:17:52.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0712e-05 2024-09-13T09:17:52.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973685+0000", 2024-09-13T09:17:52.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9029999999999998e-06 2024-09-13T09:17:52.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973690+0000", 2024-09-13T09:17:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.955e-06 2024-09-13T09:17:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998613+0000", 2024-09-13T09:17:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024922546 2024-09-13T09:17:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998695+0000", 2024-09-13T09:17:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2066999999999999e-05 2024-09-13T09:17:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998789+0000", 2024-09-13T09:17:52.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3813000000000005e-05 2024-09-13T09:17:52.455 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.455 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.455 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.456 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4111 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.456 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:17:52.456 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4180509780000001, 2024-09-13T09:17:52.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025376396999999998, 2024-09-13T09:17:52.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.456 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.456 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.456 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.457 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4111 2024-09-13T09:17:52.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.457 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:17:52.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:17:52.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973694+0000", 2024-09-13T09:17:52.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973697+0000", 2024-09-13T09:17:52.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.384e-06 2024-09-13T09:17:52.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973703+0000", 2024-09-13T09:17:52.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9970000000000004e-06 2024-09-13T09:17:52.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973707+0000", 2024-09-13T09:17:52.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7169999999999999e-06 2024-09-13T09:17:52.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998817+0000", 2024-09-13T09:17:52.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025109446000000001 2024-09-13T09:17:52.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998898+0000", 2024-09-13T09:17:52.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1003999999999999e-05 2024-09-13T09:17:52.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999071+0000", 2024-09-13T09:17:52.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000173337 2024-09-13T09:17:52.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.463 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.464 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4112 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.464 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:17:52.464 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4180341540000008, 2024-09-13T09:17:52.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025596280999999999, 2024-09-13T09:17:52.464 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.465 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.465 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.465 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.465 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.465 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4112 2024-09-13T09:17:52.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.465 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:17:52.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:17:52.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973711+0000", 2024-09-13T09:17:52.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973713+0000", 2024-09-13T09:17:52.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2670000000000001e-06 2024-09-13T09:17:52.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973719+0000", 2024-09-13T09:17:52.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9379999999999997e-06 2024-09-13T09:17:52.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973724+0000", 2024-09-13T09:17:52.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6809999999999997e-06 2024-09-13T09:17:52.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999115+0000", 2024-09-13T09:17:52.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025390511000000001 2024-09-13T09:17:52.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999202+0000", 2024-09-13T09:17:52.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7787999999999994e-05 2024-09-13T09:17:52.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999308+0000", 2024-09-13T09:17:52.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105568 2024-09-13T09:17:52.471 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.472 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.472 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4113 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 430080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.472 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:17:52.472 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4180170089999997, 2024-09-13T09:17:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025814090000000001, 2024-09-13T09:17:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4113 2024-09-13T09:17:52.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:17:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:17:52.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973728+0000", 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973738+0000", 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4339999999999998e-06 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973745+0000", 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9639999999999997e-06 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973754+0000", 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4760000000000004e-06 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999360+0000", 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025606035999999999 2024-09-13T09:17:52.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999443+0000", 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3393000000000006e-05 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999543+0000", 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9351000000000005e-05 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4114 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4179871540000004, 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026036354000000001, 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4114 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.478 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973758+0000", 2024-09-13T09:17:52.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973761+0000", 2024-09-13T09:17:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-13T09:17:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973767+0000", 2024-09-13T09:17:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6209999999999998e-06 2024-09-13T09:17:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973772+0000", 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999593+0000", 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025820532 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999691+0000", 2024-09-13T09:17:52.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8701000000000002e-05 2024-09-13T09:17:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999795+0000", 2024-09-13T09:17:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103467 2024-09-13T09:17:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4115 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.482 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4179694109999996, 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026260549000000001, 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4115 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:17:52.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973778+0000", 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.091e-06 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973784+0000", 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.268e-06 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973789+0000", 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4599999999999996e-06 2024-09-13T09:17:52.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999836+0000", 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026046980000000001 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999932+0000", 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5863e-05 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000037+0000", 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105264 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4116 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4179530230000008, 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026488986999999999, 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.487 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4116 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973792+0000", 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973794+0000", 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8950000000000001e-06 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973800+0000", 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9780000000000002e-06 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973811+0000", 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1192999999999999e-05 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000081+0000", 2024-09-13T09:17:52.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026269942000000001 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000175+0000", 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3819000000000004e-05 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000282+0000", 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106507 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4117 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:17:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4179294789999997, 2024-09-13T09:17:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026706133999999999, 2024-09-13T09:17:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4117 2024-09-13T09:17:52.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.493 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:17:52.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:17:52.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973818+0000", 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6259999999999999e-06 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973825+0000", 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.737e-06 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973830+0000", 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4399999999999998e-06 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000317+0000", 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026486948 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000398+0000", 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1160000000000002e-05 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000522+0000", 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012466800000000001 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4118 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4179119500000006, 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026891327, 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4118 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973833+0000", 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973836+0000", 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2350000000000002e-06 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973841+0000", 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7230000000000001e-06 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973846+0000", 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1829999999999996e-06 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000563+0000", 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026717797000000001 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000630+0000", 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6152e-05 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000725+0000", 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5601999999999997e-05 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4119 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4178960119999999, 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027064775999999999, 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4119 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973849+0000", 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973852+0000", 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973859+0000", 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7589999999999997e-06 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973864+0000", 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8119999999999998e-06 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000747+0000", 2024-09-13T09:17:52.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026882653999999999 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000816+0000", 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8841e-05 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000915+0000", 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8933999999999998e-05 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.505 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14583 3.1f 3:feba6951:::rbd_data.11853ecd5f5f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.8577153360000001, 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.050883949999999997, 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14583 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:52.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534028+0000", 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538304+0000", 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00427634 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538342+0000", 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7567000000000001e-05 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538357+0000", 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4936999999999999e-05 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538411+0000", 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4166000000000003e-05 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538539+0000", 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000127966 2024-09-13T09:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.539016+0000", 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477014 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.547890+0000", 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0088736430000000005 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584825+0000", 2024-09-13T09:17:52.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036934966 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584866+0000", 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1514000000000003e-05 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584914+0000", 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8167000000000001e-05 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:52.512 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:52.512 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:52.513 INFO:teuthology.orchestra.run.smithi032.stdout:1420 punch from 0xba9686f to 0xba9b799, (0x4f2a bytes) 2024-09-13T09:17:52.513 INFO:teuthology.orchestra.run.smithi032.stdout:1421 read 0xa575ab1 thru 0xa57ce1d (0x736d bytes) 2024-09-13T09:17:52.513 INFO:teuthology.orchestra.run.smithi032.stdout:1423 trunc from 0xc7d1975 to 0xaec4d40 2024-09-13T09:17:52.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:17:52.515 INFO:teuthology.orchestra.run.smithi032.stdout:1424 writesame 0xee4b65e thru 0xee507a7 data_size 0x5(0x514a bytes) 2024-09-13T09:17:52.515 INFO:teuthology.orchestra.run.smithi032.stdout:1425 writesame 0x3efb01c thru 0x3f0a7c2 data_size 0x7(0xf7a7 bytes) 2024-09-13T09:17:52.515 INFO:teuthology.orchestra.run.smithi032.stdout:1426 read 0xe473aa1 thru 0xe473f77 (0x4d7 bytes) 2024-09-13T09:17:52.515 INFO:teuthology.orchestra.run.smithi032.stdout:1427 read 0x315d62d thru 0x3165013 (0x79e7 bytes) 2024-09-13T09:17:52.515 INFO:teuthology.orchestra.run.smithi032.stdout:1429 write 0xc178584 thru 0xc183f53 (0xb9d0 bytes) 2024-09-13T09:17:52.515 INFO:teuthology.orchestra.run.smithi032.stdout:1430 trunc from 0xee507a8 to 0x7c55caf 2024-09-13T09:17:52.515 INFO:teuthology.orchestra.run.smithi032.stdout:1431 write 0x170c05d thru 0x17170dc (0xb080 bytes) 2024-09-13T09:17:52.515 INFO:teuthology.orchestra.run.smithi032.stdout:1432 writesame 0x3d39683 thru 0x3d3ffdd data_size 0x1(0x695b bytes) 2024-09-13T09:17:52.516 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:52.508+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:52.526 INFO:teuthology.orchestra.run.smithi032.stdout:1433 punch from 0x6c34417 to 0x6c38778, (0x4361 bytes) 2024-09-13T09:17:52.546 INFO:teuthology.orchestra.run.smithi032.stdout:1434 write 0xc101463 thru 0xc10298f (0x152d bytes) 2024-09-13T09:17:52.552 INFO:teuthology.orchestra.run.smithi032.stdout:1435 punch from 0x46dd1bd to 0x46e3517, (0x635a bytes) 2024-09-13T09:17:52.557 INFO:teuthology.orchestra.run.smithi032.stdout:1436 trunc from 0xc102990 to 0xe52a29f 2024-09-13T09:17:52.566 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:52.583 INFO:teuthology.orchestra.run.smithi032.stdout:1439 trunc from 0xe52a29f to 0x5684945 2024-09-13T09:17:52.606 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:52.606 INFO:teuthology.orchestra.run.smithi032.stdout:1444 read 0xd3fa7f thru 0xd48e90 (0x9412 bytes) 2024-09-13T09:17:52.607 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:52.614 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:52.612+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:52.625 INFO:teuthology.orchestra.run.smithi032.stdout:1446 write 0x985939f thru 0x985caaf (0x3711 bytes) 2024-09-13T09:17:52.652 INFO:teuthology.orchestra.run.smithi032.stdout:1447 writesame 0xe9117ba thru 0xe91a888 data_size 0x9(0x90cf bytes) 2024-09-13T09:17:52.652 INFO:teuthology.orchestra.run.smithi032.stdout:1448 writesame 0x9b063a9 thru 0x9b0e6ef data_size 0x1(0x8347 bytes) 2024-09-13T09:17:52.681 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:52.682 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:52.696 INFO:teuthology.orchestra.run.smithi032.stdout:1449 trunc from 0xe91a889 to 0xda72f8b 2024-09-13T09:17:52.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:17:52.708 INFO:teuthology.orchestra.run.smithi032.stdout:1450 write 0x37092a3 thru 0x3718670 (0xf3ce bytes) 2024-09-13T09:17:52.734 INFO:teuthology.orchestra.run.smithi032.stdout:1452 writesame 0x10b0169 thru 0x10bb28e data_size 0x19(0xb126 bytes) 2024-09-13T09:17:52.735 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:52.732+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:52.738 INFO:teuthology.orchestra.run.smithi032.stdout:1453 writesame 0x744179c thru 0x7444a97 data_size 0x1a(0x32fc bytes) 2024-09-13T09:17:52.744 INFO:teuthology.orchestra.run.smithi032.stdout:1455 punch from 0x5305ae0 to 0x5305eec, (0x40c bytes) 2024-09-13T09:17:52.744 INFO:teuthology.orchestra.run.smithi032.stdout:1456 write 0x355f77f thru 0x356c0b4 (0xc936 bytes) 2024-09-13T09:17:52.747 INFO:teuthology.orchestra.run.smithi032.stdout:1457 write 0x87c9e0b thru 0x87d83df (0xe5d5 bytes) 2024-09-13T09:17:52.752 INFO:teuthology.orchestra.run.smithi032.stdout:1460 write 0x6c4159e thru 0x6c468b0 (0x5313 bytes) 2024-09-13T09:17:52.758 INFO:teuthology.orchestra.run.smithi032.stdout:1461 read 0x3857dd9 thru 0x38677ad (0xf9d5 bytes) 2024-09-13T09:17:52.759 INFO:teuthology.orchestra.run.smithi032.stdout:1462 write 0xd01a992 thru 0xd029e79 (0xf4e8 bytes) 2024-09-13T09:17:52.765 INFO:teuthology.orchestra.run.smithi032.stdout:1463 read 0x63a2a92 thru 0x63a5090 (0x25ff bytes) 2024-09-13T09:17:52.766 INFO:teuthology.orchestra.run.smithi032.stdout:1465 write 0xaa4156d thru 0xaa4f83b (0xe2cf bytes) 2024-09-13T09:17:52.769 INFO:teuthology.orchestra.run.smithi032.stdout:1466 write 0xb91882e thru 0xb91eb87 (0x635a bytes) 2024-09-13T09:17:52.771 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:52.771 INFO:teuthology.orchestra.run.smithi032.stdout:1468 write 0x3fa3df8 thru 0x3fb39eb (0xfbf4 bytes) 2024-09-13T09:17:52.774 INFO:teuthology.orchestra.run.smithi032.stdout:1470 read 0xade9175 thru 0xadea533 (0x13bf bytes) 2024-09-13T09:17:52.774 INFO:teuthology.orchestra.run.smithi032.stdout:1471 write 0x29374d6 thru 0x293b45c (0x3f87 bytes) 2024-09-13T09:17:52.778 INFO:teuthology.orchestra.run.smithi032.stdout:1472 read 0xbc6426 thru 0xbcd0fa (0x6cd5 bytes) 2024-09-13T09:17:52.778 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:52.791 INFO:teuthology.orchestra.run.smithi032.stdout:1474 trunc from 0xda72f8b to 0x5202d1e 2024-09-13T09:17:52.812 INFO:teuthology.orchestra.run.smithi032.stdout:1475 punch from 0x4633344 to 0x4641fd7, (0xec93 bytes) 2024-09-13T09:17:52.815 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:52.812+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:52.822 INFO:teuthology.orchestra.run.smithi032.stdout:1478 write 0x5b1df0e thru 0x5b24cb1 (0x6da4 bytes) 2024-09-13T09:17:52.825 INFO:teuthology.orchestra.run.smithi032.stdout:1479 read 0x1dee88d thru 0x1df0a3c (0x21b0 bytes) 2024-09-13T09:17:52.829 INFO:teuthology.orchestra.run.smithi032.stdout:1480 trunc from 0x5b24cb2 to 0x77b7dc0 2024-09-13T09:17:52.841 INFO:teuthology.orchestra.run.smithi032.stdout:1481 write 0x15e8943 thru 0x15f30b1 (0xa76f bytes) 2024-09-13T09:17:52.842 INFO:teuthology.orchestra.run.smithi032.stdout:1482 write 0x63fa3a5 thru 0x63faa51 (0x6ad bytes) 2024-09-13T09:17:52.844 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:52.844 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:52.844 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:52.844 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:52.844 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:52.845 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:52.845 INFO:teuthology.orchestra.run.smithi032.stdout:1485 clone 9 order 24 su 524288 sc 15 2024-09-13T09:17:52.860 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:17:52.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:17:52.917 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:52.910+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:53.017 INFO:teuthology.orchestra.run.smithi032.stdout:leaving image image_client.0-clone8 intact 2024-09-13T09:17:53.018 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:53.014+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:53.033 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:53.033 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:53.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:53.033 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:53.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.034 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5478 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.034 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:17:53.034 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4947777650000003, 2024-09-13T09:17:53.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027749532, 2024-09-13T09:17:53.034 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5478 2024-09-13T09:17:53.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.035 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:17:53.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:17:53.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:17:53.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539236+0000", 2024-09-13T09:17:53.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.041e-06 2024-09-13T09:17:53.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539242+0000", 2024-09-13T09:17:53.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0650000000000004e-06 2024-09-13T09:17:53.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539246+0000", 2024-09-13T09:17:53.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1500000000000001e-06 2024-09-13T09:17:53.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566770+0000", 2024-09-13T09:17:53.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027523178999999998 2024-09-13T09:17:53.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566846+0000", 2024-09-13T09:17:53.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5941999999999999e-05 2024-09-13T09:17:53.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566984+0000", 2024-09-13T09:17:53.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000138486 2024-09-13T09:17:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5479 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:17:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4947615009999993, 2024-09-13T09:17:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027937460000000001, 2024-09-13T09:17:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.039 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5479 2024-09-13T09:17:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:17:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:17:53.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539250+0000", 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539256+0000", 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539263+0000", 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3779999999999998e-06 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539273+0000", 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0457e-05 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567010+0000", 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027736878 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567083+0000", 2024-09-13T09:17:53.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3187000000000002e-05 2024-09-13T09:17:53.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567188+0000", 2024-09-13T09:17:53.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010491499999999999 2024-09-13T09:17:53.047 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.047 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.047 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.047 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5480 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.048 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:17:53.048 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4947342829999997, 2024-09-13T09:17:53.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028132641999999999, 2024-09-13T09:17:53.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.048 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.048 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.048 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.048 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.048 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5480 2024-09-13T09:17:53.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.048 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539280+0000", 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539297+0000", 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7147999999999999e-05 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539303+0000", 2024-09-13T09:17:53.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9039999999999997e-06 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567230+0000", 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027927250000000001 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567316+0000", 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5970999999999994e-05 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567411+0000", 2024-09-13T09:17:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4523000000000003e-05 2024-09-13T09:17:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5481 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:17:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4947042869999994, 2024-09-13T09:17:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028317978000000001, 2024-09-13T09:17:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5481 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:17:53.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539307+0000", 2024-09-13T09:17:53.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539311+0000", 2024-09-13T09:17:53.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.264e-06 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539317+0000", 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539322+0000", 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6949999999999999e-06 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567446+0000", 2024-09-13T09:17:53.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028124341000000001 2024-09-13T09:17:53.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567527+0000", 2024-09-13T09:17:53.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0785000000000007e-05 2024-09-13T09:17:53.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567626+0000", 2024-09-13T09:17:53.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9103000000000003e-05 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5482 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4946864620000007, 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028619326, 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5482 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:17:53.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539325+0000", 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539333+0000", 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.055e-06 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539340+0000", 2024-09-13T09:17:53.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4980000000000003e-06 2024-09-13T09:17:53.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539344+0000", 2024-09-13T09:17:53.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5369999999999999e-06 2024-09-13T09:17:53.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567655+0000", 2024-09-13T09:17:53.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028310906 2024-09-13T09:17:53.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567732+0000", 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6829999999999995e-05 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567945+0000", 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000212909 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5483 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4946632009999998, 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028784015999999999, 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5483 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539351+0000", 2024-09-13T09:17:53.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-13T09:17:53.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539357+0000", 2024-09-13T09:17:53.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0090000000000004e-06 2024-09-13T09:17:53.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539361+0000", 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.357e-06 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567974+0000", 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028612385000000001 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568050+0000", 2024-09-13T09:17:53.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5820000000000003e-05 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568133+0000", 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3551e-05 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5484 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:17:53.066 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4946466489999999, 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028949923999999998, 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5484 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539365+0000", 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539367+0000", 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539374+0000", 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539378+0000", 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568159+0000", 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028780490999999998 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568236+0000", 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7156000000000005e-05 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568316+0000", 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9598999999999995e-05 2024-09-13T09:17:53.070 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.071 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5485 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.071 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:17:53.071 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4938267700000001, 2024-09-13T09:17:53.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028332593, 2024-09-13T09:17:53.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.071 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.071 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5485 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:17:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540189+0000", 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540198+0000", 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9099999999999994e-06 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540206+0000", 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568360+0000", 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028154324000000001 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568436+0000", 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5821000000000005e-05 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568518+0000", 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1895000000000001e-05 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.075 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5486 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.076 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:17:53.076 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4937737609999999, 2024-09-13T09:17:53.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028479536999999999, 2024-09-13T09:17:53.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.076 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.076 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5486 2024-09-13T09:17:53.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.076 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540241+0000", 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4290000000000001e-06 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540249+0000", 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9480000000000008e-06 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540255+0000", 2024-09-13T09:17:53.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6019999999999999e-06 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568543+0000", 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028288260999999999 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568636+0000", 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2627999999999995e-05 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568718+0000", 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2213999999999996e-05 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5487 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4937526759999997, 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028653435000000001, 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.080 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5487 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:17:53.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540259+0000", 2024-09-13T09:17:53.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540262+0000", 2024-09-13T09:17:53.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.413e-06 2024-09-13T09:17:53.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540268+0000", 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3940000000000001e-06 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540272+0000", 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3989999999999997e-06 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568745+0000", 2024-09-13T09:17:53.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028472173999999999 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568822+0000", 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7694000000000005e-05 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568913+0000", 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0779000000000006e-05 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5488 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 430080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4937352760000007, 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028788326999999999, 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5488 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.085 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:17:53.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540279+0000", 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2409999999999998e-06 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540285+0000", 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3180000000000003e-06 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540300+0000", 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4625e-05 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568939+0000", 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028639688 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569016+0000", 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6556000000000004e-05 2024-09-13T09:17:53.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569065+0000", 2024-09-13T09:17:53.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9246000000000003e-05 2024-09-13T09:17:53.089 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.089 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.089 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.089 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5489 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.089 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:17:53.089 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4937076820000001, 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028875332, 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5489 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:17:53.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540304+0000", 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540307+0000", 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4370000000000001e-06 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540313+0000", 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540318+0000", 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:17:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569079+0000", 2024-09-13T09:17:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028760714 2024-09-13T09:17:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569128+0000", 2024-09-13T09:17:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8702999999999998e-05 2024-09-13T09:17:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569180+0000", 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2105999999999999e-05 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5490 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4936891869999993, 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029041027000000001, 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5490 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540326+0000", 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9840000000000001e-06 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540332+0000", 2024-09-13T09:17:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9730000000000002e-06 2024-09-13T09:17:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540338+0000", 2024-09-13T09:17:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3550000000000001e-06 2024-09-13T09:17:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569202+0000", 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028863521 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569276+0000", 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569364+0000", 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7594000000000002e-05 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5491 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4936698029999995, 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029221864, 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5491 2024-09-13T09:17:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:17:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:17:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:17:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540350+0000", 2024-09-13T09:17:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4250000000000001e-06 2024-09-13T09:17:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540356+0000", 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2960000000000004e-06 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540361+0000", 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569388+0000", 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029026744 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569474+0000", 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6519000000000003e-05 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569564+0000", 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9969999999999994e-05 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5492 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4936465059999993, 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029401203000000001, 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5492 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540365+0000", 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540368+0000", 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540374+0000", 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9349999999999999e-06 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540378+0000", 2024-09-13T09:17:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.651e-06 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569591+0000", 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029212557 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569668+0000", 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569767+0000", 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9061000000000006e-05 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5493 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4936296420000001, 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029599424999999999, 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5493 2024-09-13T09:17:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:17:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540382+0000", 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540384+0000", 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0430000000000002e-06 2024-09-13T09:17:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540390+0000", 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540395+0000", 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5990000000000004e-06 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569795+0000", 2024-09-13T09:17:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029400582000000001 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569880+0000", 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5210999999999994e-05 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569982+0000", 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101584 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5494 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4936135569999998, 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029770637999999999, 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5494 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540398+0000", 2024-09-13T09:17:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540406+0000", 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540413+0000", 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540418+0000", 2024-09-13T09:17:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8289999999999998e-06 2024-09-13T09:17:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570007+0000", 2024-09-13T09:17:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029588851999999999 2024-09-13T09:17:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570084+0000", 2024-09-13T09:17:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7253e-05 2024-09-13T09:17:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570169+0000", 2024-09-13T09:17:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5391999999999998e-05 2024-09-13T09:17:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5495 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4935752410000003, 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029938664, 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5495 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:17:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540440+0000", 2024-09-13T09:17:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0319999999999999e-06 2024-09-13T09:17:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540446+0000", 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8549999999999996e-06 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540451+0000", 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570198+0000", 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029746060000000001 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570284+0000", 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6044999999999994e-05 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570376+0000", 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2071000000000007e-05 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5496 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4935562860000005, 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030121617999999999, 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5496 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:17:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540458+0000", 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540464+0000", 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.156e-06 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540468+0000", 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570404+0000", 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029935632 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570488+0000", 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3873000000000001e-05 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570578+0000", 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9757000000000001e-05 2024-09-13T09:17:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5497 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4935397219999995, 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030305259000000001, 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5497 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540472+0000", 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540474+0000", 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0490000000000002e-06 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540480+0000", 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1970000000000001e-06 2024-09-13T09:17:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540491+0000", 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0964e-05 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570608+0000", 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030116105000000001 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570688+0000", 2024-09-13T09:17:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0143999999999997e-05 2024-09-13T09:17:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570778+0000", 2024-09-13T09:17:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0131999999999997e-05 2024-09-13T09:17:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.131 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:53.133 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:17:53.133 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:53.126+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:53.187 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-13T09:17:53.209 INFO:teuthology.orchestra.run.smithi032.stdout:1486 writesame 0xa6f57df thru 0xa701101 data_size 0x1(0xb923 bytes) 2024-09-13T09:17:53.224 INFO:teuthology.orchestra.run.smithi032.stdout:1487 write 0x269b44b thru 0x26aa536 (0xf0ec bytes) 2024-09-13T09:17:53.234 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:53.230+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:53.235 INFO:teuthology.orchestra.run.smithi032.stdout:1489 write 0x6af3245 thru 0x6b02850 (0xf60c bytes) 2024-09-13T09:17:53.239 INFO:teuthology.orchestra.run.smithi032.stdout:1490 read 0x2633207 thru 0x263fef6 (0xccf0 bytes) 2024-09-13T09:17:53.239 INFO:teuthology.orchestra.run.smithi032.stdout:1491 punch from 0x687cc91 to 0x688acad, (0xe01c bytes) 2024-09-13T09:17:53.239 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:53.240 INFO:teuthology.orchestra.run.smithi032.stdout:1493 writesame 0x1602c15 thru 0x160f941 data_size 0x37(0xcd2d bytes) 2024-09-13T09:17:53.249 INFO:teuthology.orchestra.run.smithi032.stdout:1494 read 0xc198f6 thru 0xc1fbc3 (0x62ce bytes) 2024-09-13T09:17:53.249 INFO:teuthology.orchestra.run.smithi032.stdout:1496 punch from 0x4bce3e5 to 0x4bd6d4c, (0x8967 bytes) 2024-09-13T09:17:53.249 INFO:teuthology.orchestra.run.smithi032.stdout:1497 write 0x31fb5ff thru 0x3202a9c (0x749e bytes) 2024-09-13T09:17:53.259 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16837 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [call rbd.sparse_copyup in=173207b,set-alloc-hint object_size 16777216 write_size 16777216,write 3145728~10909 in=10909b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:17:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:17:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.001476743, 2024-09-13T09:17:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001502001, 2024-09-13T09:17:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:17:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16837 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260121+0000", 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260266+0000", 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014495999999999999 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260287+0000", 2024-09-13T09:17:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1336e-05 2024-09-13T09:17:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260296+0000", 2024-09-13T09:17:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6170000000000007e-06 2024-09-13T09:17:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260352+0000", 2024-09-13T09:17:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6252999999999998e-05 2024-09-13T09:17:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260520+0000", 2024-09-13T09:17:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000168381 2024-09-13T09:17:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:17:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260807+0000", 2024-09-13T09:17:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028624100000000001 2024-09-13T09:17:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.264 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:17:53.265 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 1 2024-09-13T09:17:53.265 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:53.265 INFO:teuthology.orchestra.run.smithi032.stdout:1499 write 0x9d39dcb thru 0x9d4181c (0x7a52 bytes) 2024-09-13T09:17:53.266 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:53.266 INFO:teuthology.orchestra.run.smithi032.stdout:1502 read 0x1af76b4 thru 0x1af952a (0x1e77 bytes) 2024-09-13T09:17:53.266 INFO:teuthology.orchestra.run.smithi032.stdout:1503 read 0x3ce5854 thru 0x3ce77c5 (0x1f72 bytes) 2024-09-13T09:17:53.266 INFO:teuthology.orchestra.run.smithi032.stdout:1504 writesame 0x205b665 thru 0x2067458 data_size 0x2(0xbdf4 bytes) 2024-09-13T09:17:53.275 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:17:53.299 INFO:teuthology.orchestra.run.smithi032.stdout:1505 read 0x9209ada thru 0x92164c2 (0xc9e9 bytes) 2024-09-13T09:17:53.299 INFO:teuthology.orchestra.run.smithi032.stdout:1506 read 0x562f9a5 thru 0x56305be (0xc1a bytes) 2024-09-13T09:17:53.300 INFO:teuthology.orchestra.run.smithi032.stdout:1507 write 0x49cda8 thru 0x4a5750 (0x89a9 bytes) 2024-09-13T09:17:53.303 INFO:teuthology.orchestra.run.smithi032.stdout:1509 write 0x58deedd thru 0x58eccd4 (0xddf8 bytes) 2024-09-13T09:17:53.314 INFO:teuthology.orchestra.run.smithi032.stdout:1511 read 0x753e607 thru 0x7544306 (0x5d00 bytes) 2024-09-13T09:17:53.315 INFO:teuthology.orchestra.run.smithi032.stdout:1512 write 0x6105b5e thru 0x61098d2 (0x3d75 bytes) 2024-09-13T09:17:53.332 INFO:teuthology.orchestra.run.smithi032.stdout:1516 writesame 0xe30ede7 thru 0xe3144e9 data_size 0x63(0x5703 bytes) 2024-09-13T09:17:53.334 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:53.332+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:53.335 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:53.335 INFO:teuthology.orchestra.run.smithi032.stdout:1518 write 0x286d7aa thru 0x2874891 (0x70e8 bytes) 2024-09-13T09:17:53.345 INFO:teuthology.orchestra.run.smithi032.stdout:1520 read 0x2586c0f thru 0x258c48c (0x587e bytes) 2024-09-13T09:17:53.354 INFO:teuthology.orchestra.run.smithi032.stdout:1521 trunc from 0xe3144ea to 0x83b63e1 2024-09-13T09:17:53.397 INFO:teuthology.orchestra.run.smithi032.stdout:1522 write 0x5e073e9 thru 0x5e11985 (0xa59d bytes) 2024-09-13T09:17:53.399 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:53.402 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:53.403 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:53.403 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:53.403 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:53.403 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:53.406 INFO:teuthology.orchestra.run.smithi032.stdout:1527 trunc from 0x83b63e1 to 0x3b2509a 2024-09-13T09:17:53.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:17:53.423 INFO:teuthology.orchestra.run.smithi032.stdout:1531 read 0x1d8a482 thru 0x1d8ef60 (0x4adf bytes) 2024-09-13T09:17:53.435 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:53.430+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:53.446 INFO:teuthology.orchestra.run.smithi032.stdout:1532 write 0xbbb4327 thru 0xbbb9abe (0x5798 bytes) 2024-09-13T09:17:53.448 INFO:teuthology.orchestra.run.smithi032.stdout:1533 clone 10 order 24 su 4194304 sc 14 2024-09-13T09:17:53.536 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:53.532+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:53.550 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:53.550 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:53.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:53.551 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:53.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.551 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:1545 3.9 e29/22 3:94e462d4:::rbd_object_map.1185192b47c4:head v 29'138, mlcod=29'136)", 2024-09-13T09:17:53.551 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:34.600334+0000", 2024-09-13T09:17:53.551 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.948396691999999, 2024-09-13T09:17:53.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0026055739999999998, 2024-09-13T09:17:53.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.551 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.551 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600334+0000", 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600334+0000", 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600332+0000", 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600350+0000", 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8450000000000001e-05 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600385+0000", 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4839000000000003e-05 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600404+0000", 2024-09-13T09:17:53.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8876999999999999e-05 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600499+0000", 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5008999999999998e-05 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.600541+0000", 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2015999999999999e-05 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.602891+0000", 2024-09-13T09:17:53.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0023506579999999998 2024-09-13T09:17:53.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:34.602939+0000", 2024-09-13T09:17:53.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7935000000000001e-05 2024-09-13T09:17:53.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.555 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.556 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:2893 3.9 e31/22 3:9571f390:::rbd_data.118591b29fad.00000000000000a0:head v 31'169, mlcod=31'166)", 2024-09-13T09:17:53.556 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:38.299390+0000", 2024-09-13T09:17:53.556 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.249340140999999, 2024-09-13T09:17:53.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025344059999999999, 2024-09-13T09:17:53.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.556 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.556 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.299390+0000", 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.299390+0000", 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.299390+0000", 2024-09-13T09:17:53.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.299399+0000", 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7510000000000003e-06 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.299417+0000", 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7220000000000001e-05 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.299428+0000", 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1154e-05 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.300006+0000", 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00057788100000000003 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.300067+0000", 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1828000000000005e-05 2024-09-13T09:17:53.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.301872+0000", 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001804784 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.301925+0000", 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2541999999999999e-05 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3022 3.9 e31/22 3:91681905:::rbd_data.118591b29fad.0000000000000209:head v 31'171, mlcod=31'169)", 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.080490730999999, 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002956209, 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468237+0000", 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468300+0000", 2024-09-13T09:17:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3202999999999998e-05 2024-09-13T09:17:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468353+0000", 2024-09-13T09:17:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2735999999999998e-05 2024-09-13T09:17:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468374+0000", 2024-09-13T09:17:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0930000000000001e-05 2024-09-13T09:17:53.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468568+0000", 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000194262 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468627+0000", 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8743000000000003e-05 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.471132+0000", 2024-09-13T09:17:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025055540000000001 2024-09-13T09:17:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.471196+0000", 2024-09-13T09:17:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3590999999999995e-05 2024-09-13T09:17:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.565 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "pg_update_log_missing(3.4 epoch 35/22 rep_tid 113 entries 35'31 (0'0) error 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head by client.4485.0:3311 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 31'29)", 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 11.020810217999999, 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033204409999999999, 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:17:53.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:17:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527918+0000", 2024-09-13T09:17:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527943+0000", 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4752999999999998e-05 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527988+0000", 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4693000000000001e-05 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528029+0000", 2024-09-13T09:17:53.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1155999999999997e-05 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528194+0000", 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164792 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.531241+0000", 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003046867 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "pg_update_log_missing(3.4 epoch 35/22 rep_tid 116 entries 35'33 (0'0) error 3:24ec887b:::rbd_data.118591b29fad.000000000000007c:head by client.4485.0:3321 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 35'30)", 2024-09-13T09:17:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.534125+0000", 2024-09-13T09:17:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 11.014605029, 2024-09-13T09:17:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027308850000000002, 2024-09-13T09:17:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:17:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534125+0000", 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534125+0000", 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534124+0000", 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534148+0000", 2024-09-13T09:17:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4233e-05 2024-09-13T09:17:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534236+0000", 2024-09-13T09:17:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7832999999999997e-05 2024-09-13T09:17:53.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534272+0000", 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6195999999999997e-05 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.534431+0000", 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000159362 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536856+0000", 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024251210000000001 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3323 3.4 e35/22 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head v 35'34, mlcod=35'31)", 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 11.012935292, 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037621640000000001, 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:17:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:17:53.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535793+0000", 2024-09-13T09:17:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535811+0000", 2024-09-13T09:17:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8170000000000001e-05 2024-09-13T09:17:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535840+0000", 2024-09-13T09:17:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9105999999999999e-05 2024-09-13T09:17:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535877+0000", 2024-09-13T09:17:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7017000000000002e-05 2024-09-13T09:17:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536001+0000", 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012342899999999999 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536073+0000", 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2123e-05 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539433+0000", 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033596630000000001 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539557+0000", 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012470600000000001 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3358 3.4 e35/22 3:26bebe25:::rbd_data.118591b29fad.0000000000000095:head v 35'37, mlcod=35'34)", 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.553104+0000", 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.995626006, 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024175500000000001, 2024-09-13T09:17:53.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553104+0000", 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553104+0000", 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553102+0000", 2024-09-13T09:17:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553124+0000", 2024-09-13T09:17:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1977e-05 2024-09-13T09:17:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553161+0000", 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7171e-05 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553181+0000", 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.916e-05 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553307+0000", 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012668600000000001 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.553356+0000", 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000001e-05 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.555437+0000", 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020812780000000002 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.555522+0000", 2024-09-13T09:17:53.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5154000000000005e-05 2024-09-13T09:17:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3364 3.4 e35/22 3:2748ed7f:::rbd_data.118591b29fad.0000000000000215:head v 35'39, mlcod=35'35)", 2024-09-13T09:17:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.560115+0000", 2024-09-13T09:17:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.988615803, 2024-09-13T09:17:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0026426610000000001, 2024-09-13T09:17:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.584 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560115+0000", 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560115+0000", 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560113+0000", 2024-09-13T09:17:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560130+0000", 2024-09-13T09:17:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7163000000000002e-05 2024-09-13T09:17:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560164+0000", 2024-09-13T09:17:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4196999999999998e-05 2024-09-13T09:17:53.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560186+0000", 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1908000000000001e-05 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560346+0000", 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015956400000000001 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.560392+0000", 2024-09-13T09:17:53.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6113999999999999e-05 2024-09-13T09:17:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.562667+0000", 2024-09-13T09:17:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002274501 2024-09-13T09:17:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.562757+0000", 2024-09-13T09:17:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0802000000000002e-05 2024-09-13T09:17:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.588 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:15977 3.4 3:27c24f46:::rbd_data.11856af9a4d3.0000000000000005:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:17:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:17:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5536657790000001, 2024-09-13T09:17:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003633163, 2024-09-13T09:17:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15977 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995063+0000", 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995071+0000", 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3930000000000002e-06 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995086+0000", 2024-09-13T09:17:53.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4970999999999999e-05 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995095+0000", 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8810000000000002e-06 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995180+0000", 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5012000000000005e-05 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995378+0000", 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019788600000000001 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.998698+0000", 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003320267 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16010 3.4 3:27c24f46:::rbd_data.11856af9a4d3.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5480569870000001, 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037539579999999999, 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16010 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000672+0000", 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000679+0000", 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9120000000000001e-06 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000693+0000", 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3745e-05 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000702+0000", 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.064e-06 2024-09-13T09:17:53.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000763+0000", 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1309000000000005e-05 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000877+0000", 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114214 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.001145+0000", 2024-09-13T09:17:53.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000267765 2024-09-13T09:17:53.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:53.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.002514+0000", 2024-09-13T09:17:53.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013688730000000001 2024-09-13T09:17:53.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:53.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004301+0000", 2024-09-13T09:17:53.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017869839999999999 2024-09-13T09:17:53.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004380+0000", 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8861000000000003e-05 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004427+0000", 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7713000000000002e-05 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.599 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16047 3.0 3:002d813e:::rbd_data.11856af9a4d3.0000000000000014:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5391799850000001, 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030667929999999999, 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16047 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009548+0000", 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:53.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009559+0000", 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0936e-05 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009579+0000", 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9976e-05 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009590+0000", 2024-09-13T09:17:53.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1228e-05 2024-09-13T09:17:53.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009670+0000", 2024-09-13T09:17:53.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9765999999999995e-05 2024-09-13T09:17:53.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009776+0000", 2024-09-13T09:17:53.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106049 2024-09-13T09:17:53.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:17:53.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009991+0000", 2024-09-13T09:17:53.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021494999999999999 2024-09-13T09:17:53.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:53.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.011621+0000", 2024-09-13T09:17:53.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016294650000000001 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012469+0000", 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000848598 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012551+0000", 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2144999999999994e-05 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012617+0000", 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5827999999999994e-05 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16226 3.4 3:22419c10:::rbd_data.1185f20bdb42.0000000000000013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3201798810000001, 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033492510000000001, 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.606 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16226 2024-09-13T09:17:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:17:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228546+0000", 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999957 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228569+0000", 2024-09-13T09:17:53.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2347000000000001e-05 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228618+0000", 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9509999999999999e-05 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228643+0000", 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4759000000000001e-05 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228738+0000", 2024-09-13T09:17:53.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5273999999999996e-05 2024-09-13T09:17:53.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228819+0000", 2024-09-13T09:17:53.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0623000000000004e-05 2024-09-13T09:17:53.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:53.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228921+0000", 2024-09-13T09:17:53.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010215199999999999 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.230000+0000", 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010793510000000001 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231718+0000", 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001717873 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231813+0000", 2024-09-13T09:17:53.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4473000000000002e-05 2024-09-13T09:17:53.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231900+0000", 2024-09-13T09:17:53.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7158999999999997e-05 2024-09-13T09:17:53.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.612 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.613 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16293 3.4 3:22419c10:::rbd_data.1185f20bdb42.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:17:53.613 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:17:53.613 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.1665692000000001, 2024-09-13T09:17:53.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015655326000000001, 2024-09-13T09:17:53.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.613 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16293 2024-09-13T09:17:53.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.613 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:17:53.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:17:53.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382158+0000", 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382176+0000", 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7863000000000002e-05 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382211+0000", 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.481e-05 2024-09-13T09:17:53.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382226+0000", 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.537e-05 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382354+0000", 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012803599999999999 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382591+0000", 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023629099999999999 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.385269+0000", 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002678532 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397515+0000", 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012245946000000001 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397551+0000", 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5856999999999999e-05 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397699+0000", 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148255 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397817+0000", 2024-09-13T09:17:53.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117299 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4485.0:16293 3.4 e42/41 ondisk, result = 0)", 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.1562376599999999, 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0050343319999999999, 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:17:53.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:17:53.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392491+0000", 2024-09-13T09:17:53.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:53.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392500+0000", 2024-09-13T09:17:53.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8300000000000002e-06 2024-09-13T09:17:53.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392529+0000", 2024-09-13T09:17:53.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9260000000000001e-05 2024-09-13T09:17:53.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392542+0000", 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2969999999999999e-05 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397468+0000", 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0049256880000000001 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397496+0000", 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8209999999999999e-05 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397527+0000", 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1276999999999999e-05 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16295 3.4 3:22419c10:::rbd_data.1185f20bdb42.0000000000000013:head [truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:17:53.623 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.402877+0000", 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.1458538030000001, 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025515070000000002, 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16295 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.402877+0000", 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.402877+0000", 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.402873+0000", 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.402896+0000", 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2762999999999999e-05 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.402936+0000", 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9860000000000001e-05 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.402955+0000", 2024-09-13T09:17:53.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8967e-05 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.403028+0000", 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2763999999999997e-05 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.403144+0000", 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011617599999999999 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.403294+0000", 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015067699999999999 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.404815+0000", 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015205500000000001 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.405353+0000", 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000537796 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.405396+0000", 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3442999999999997e-05 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.405428+0000", 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1918000000000002e-05 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.629 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16592 3.0 3:03995c04:::rbd_id.image_client.0-clone9:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.981036+0000", 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.56769444300000005, 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024380880000000001, 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16592 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.981036+0000", 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.981036+0000", 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.981033+0000", 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.981048+0000", 2024-09-13T09:17:53.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4078e-05 2024-09-13T09:17:53.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.981071+0000", 2024-09-13T09:17:53.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3932999999999999e-05 2024-09-13T09:17:53.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.981081+0000", 2024-09-13T09:17:53.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0009e-05 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.981140+0000", 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8461999999999997e-05 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.981274+0000", 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013400400000000001 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.981401+0000", 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000127403 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.982439+0000", 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00103765 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.983370+0000", 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00093150500000000001 2024-09-13T09:17:53.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.983426+0000", 2024-09-13T09:17:53.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-05 2024-09-13T09:17:53.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.983474+0000", 2024-09-13T09:17:53.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8542999999999999e-05 2024-09-13T09:17:53.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.635 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16837 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [call rbd.sparse_copyup in=173207b,set-alloc-hint object_size 16777216 write_size 16777216,write 3145728~10909 in=10909b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.288606847, 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004073887, 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16837 2024-09-13T09:17:53.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.637 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:17:53.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:17:53.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260121+0000", 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260266+0000", 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014495999999999999 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260287+0000", 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1336e-05 2024-09-13T09:17:53.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260296+0000", 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6170000000000007e-06 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260352+0000", 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6252999999999998e-05 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260520+0000", 2024-09-13T09:17:53.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000168381 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260807+0000", 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028624100000000001 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.262062+0000", 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012551490000000001 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264128+0000", 2024-09-13T09:17:53.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020660380000000001 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264167+0000", 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9714e-05 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264198+0000", 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0105000000000001e-05 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.641 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16964 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [stat,zero 8912896~6815744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.358046+0000", 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.19068429100000001, 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027898210000000001, 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16964 2024-09-13T09:17:53.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358046+0000", 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358046+0000", 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358044+0000", 2024-09-13T09:17:53.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358055+0000", 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0958000000000001e-05 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358081+0000", 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5978000000000001e-05 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358092+0000", 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.116e-05 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358184+0000", 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2243000000000004e-05 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358320+0000", 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013623500000000001 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.360836+0000", 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025156699999999998 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:17005 3.1a e43/41 3:5e94b56b:::rbd_data.11859d3d11dd.0000000000000008:head v 43'54, mlcod=42'52)", 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.16304219, 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031754880000000002, 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.646 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:53.647 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:17:53.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:17:53.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.386997+0000", 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00130933 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387013+0000", 2024-09-13T09:17:53.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5752000000000001e-05 2024-09-13T09:17:53.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387022+0000", 2024-09-13T09:17:53.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1689999999999998e-06 2024-09-13T09:17:53.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387092+0000", 2024-09-13T09:17:53.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9385000000000001e-05 2024-09-13T09:17:53.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387127+0000", 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5061000000000002e-05 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.388821+0000", 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001694463 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.388864+0000", 2024-09-13T09:17:53.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2721e-05 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:17028 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [stat,zero 4194304~4718592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.13922373699999999, 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036807770000000001, 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:53.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 17028 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:17:53.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409504+0000", 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409515+0000", 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0485e-05 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409540+0000", 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4808e-05 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409550+0000", 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0376e-05 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409616+0000", 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6130999999999994e-05 2024-09-13T09:17:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409725+0000", 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108614 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.413188+0000", 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003462849 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:53.656 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:53.656 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:53.660 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:17:53.661 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:53.652+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:53.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:17:53.798 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:53.796+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:53.833 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:53.833 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:53.833 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:53.833 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:53.848 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:17:53.866 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-13T09:17:53.867 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:53.864+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:53.902 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:53.902 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:53.920 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:53.937 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:53.937 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:53.953 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:53.953 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:53.960 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:53.954+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:53.976 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:53.990 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:53.990 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:53.997 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:53.997 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:54.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:17:54.017 INFO:teuthology.orchestra.run.smithi032.stdout:leaving image image_client.0-clone9 intact 2024-09-13T09:17:54.021 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:54.042 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:54.042 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:54.060 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:54.056+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:54.142 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:54.143 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:54.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:17:54.161 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:54.156+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:54.262 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:54.258+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:54.295 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:54.295 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:54.298 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-13T09:17:54.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:17:54.402 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:54.398+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:54.464 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:54.458+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:54.482 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:54.482 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:54.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:17:54.565 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:54.560+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:54.598 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:54.598 INFO:teuthology.orchestra.run.smithi032.stdout:1536 punch from 0x29c7181 to 0x29c8af0, (0x196f bytes) 2024-09-13T09:17:54.604 INFO:teuthology.orchestra.run.smithi032.stdout:1537 write 0x922728e thru 0x922cc89 (0x59fc bytes) 2024-09-13T09:17:54.609 INFO:teuthology.orchestra.run.smithi032.stdout:1538 write 0x528e745 thru 0x52952d2 (0x6b8e bytes) 2024-09-13T09:17:54.627 INFO:teuthology.orchestra.run.smithi032.stdout:1540 trunc from 0xbbb9abf to 0x47443fa 2024-09-13T09:17:54.665 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:54.660+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:54.686 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:54.686 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:54.688 INFO:teuthology.orchestra.run.smithi032.stdout:1541 trunc from 0x47443fa to 0x541c9 2024-09-13T09:17:54.704 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:17:54.755 INFO:teuthology.orchestra.run.smithi032.stdout:1543 write 0x3aec891 thru 0x3af37cf (0x6f3f bytes) 2024-09-13T09:17:54.757 INFO:teuthology.orchestra.run.smithi032.stdout:1545 read 0xdeb854 thru 0xdeb87c (0x29 bytes) 2024-09-13T09:17:54.766 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:54.762+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:54.772 INFO:teuthology.orchestra.run.smithi032.stdout:1546 trunc from 0x3af37d0 to 0x9101e91 2024-09-13T09:17:54.786 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:54.806 INFO:teuthology.orchestra.run.smithi032.stdout:1548 writesame 0xc2a1349 thru 0xc2b028a data_size 0x7d(0xef42 bytes) 2024-09-13T09:17:54.806 INFO:teuthology.orchestra.run.smithi032.stdout:1549 write 0x8f719c9 thru 0x8f7ec05 (0xd23d bytes) 2024-09-13T09:17:54.822 INFO:teuthology.orchestra.run.smithi032.stdout:1550 trunc from 0xc2b028b to 0x8ddbd94 2024-09-13T09:17:54.826 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:54.826 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:54.838 INFO:teuthology.orchestra.run.smithi032.stdout:1552 punch from 0x292e5b7 to 0x2935464, (0x6ead bytes) 2024-09-13T09:17:54.838 INFO:teuthology.orchestra.run.smithi032.stdout:1553 writesame 0x102978c thru 0x103264a data_size 0xd(0x8ebf bytes) 2024-09-13T09:17:54.841 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:17:54.846 INFO:teuthology.orchestra.run.smithi032.stdout:1554 punch from 0x3aa7f0d to 0x3ab6650, (0xe743 bytes) 2024-09-13T09:17:54.846 INFO:teuthology.orchestra.run.smithi032.stdout:1555 read 0x5d66cf4 thru 0x5d75c81 (0xef8e bytes) 2024-09-13T09:17:54.847 INFO:teuthology.orchestra.run.smithi032.stdout:1558 trunc from 0x8ddbd94 to 0x83ff4d9 2024-09-13T09:17:54.863 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:54.863 INFO:teuthology.orchestra.run.smithi032.stdout:1560 writesame 0x4410994 thru 0x441bc7a data_size 0xd(0xb2e7 bytes) 2024-09-13T09:17:54.867 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:54.864+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:54.871 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:54.888 INFO:teuthology.orchestra.run.smithi032.stdout:1563 write 0xa31bd4e thru 0xa31dc21 (0x1ed4 bytes) 2024-09-13T09:17:54.895 INFO:teuthology.orchestra.run.smithi032.stdout:1565 read 0x26b8a1f thru 0x26c388e (0xae70 bytes) 2024-09-13T09:17:54.895 INFO:teuthology.orchestra.run.smithi032.stdout:1566 writesame 0x6d975f1 thru 0x6d9b853 data_size 0x37(0x4263 bytes) 2024-09-13T09:17:54.901 INFO:teuthology.orchestra.run.smithi032.stdout:1567 read 0x39ae2b1 thru 0x39b1b12 (0x3862 bytes) 2024-09-13T09:17:54.902 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:54.902 INFO:teuthology.orchestra.run.smithi032.stdout:1569 punch from 0x469b26e to 0x469d944, (0x26d6 bytes) 2024-09-13T09:17:54.923 INFO:teuthology.orchestra.run.smithi032.stdout:1570 writesame 0xd201bbf thru 0xd209268 data_size 0x7a(0x76aa bytes) 2024-09-13T09:17:54.923 INFO:teuthology.orchestra.run.smithi032.stdout:1571 punch from 0x8253bf3 to 0x825b78a, (0x7b97 bytes) 2024-09-13T09:17:54.945 INFO:teuthology.orchestra.run.smithi032.stdout:1572 trunc from 0xd209269 to 0x4e1567a 2024-09-13T09:17:54.960 INFO:teuthology.orchestra.run.smithi032.stdout:1573 write 0x32c08c5 thru 0x32cfdb8 (0xf4f4 bytes) 2024-09-13T09:17:54.963 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:54.963 INFO:teuthology.orchestra.run.smithi032.stdout:1575 read 0x1c0335d thru 0x1c12d73 (0xfa17 bytes) 2024-09-13T09:17:54.964 INFO:teuthology.orchestra.run.smithi032.stdout:1578 write 0x449e353 thru 0x44a7b8e (0x983c bytes) 2024-09-13T09:17:54.967 INFO:teuthology.orchestra.run.smithi032.stdout:1580 write 0x374381d thru 0x374f570 (0xbd54 bytes) 2024-09-13T09:17:54.968 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:54.962+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:54.970 INFO:teuthology.orchestra.run.smithi032.stdout:1582 read 0x21f9c0d thru 0x21fbfd7 (0x23cb bytes) 2024-09-13T09:17:54.970 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:54.971 INFO:teuthology.orchestra.run.smithi032.stdout:1586 punch from 0x19c05e1 to 0x19cb689, (0xb0a8 bytes) 2024-09-13T09:17:54.971 INFO:teuthology.orchestra.run.smithi032.stdout:1587 read 0x1ae0599 thru 0x1aeec57 (0xe6bf bytes) 2024-09-13T09:17:55.006 INFO:teuthology.orchestra.run.smithi032.stdout:1589 write 0xdd56307 thru 0xdd64837 (0xe531 bytes) 2024-09-13T09:17:55.009 INFO:teuthology.orchestra.run.smithi032.stdout:1591 writesame 0x854815e thru 0x854cb82 data_size 0x39(0x4a25 bytes) 2024-09-13T09:17:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:1592 read 0xa0e6ab thru 0xa12a92 (0x43e8 bytes) 2024-09-13T09:17:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:55.016 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:55.016 INFO:teuthology.orchestra.run.smithi032.stdout:1598 punch from 0x3b3383 to 0x3b865d, (0x52da bytes) 2024-09-13T09:17:55.023 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:55.023 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:55.029 INFO:teuthology.orchestra.run.smithi032.stdout:1599 write 0xe5c63eb thru 0xe5cc7e1 (0x63f7 bytes) 2024-09-13T09:17:55.036 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:55.036 INFO:teuthology.orchestra.run.smithi032.stdout:1601 punch from 0x11c82a1 to 0x11cf3e3, (0x7142 bytes) 2024-09-13T09:17:55.036 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:55.036 INFO:teuthology.orchestra.run.smithi032.stdout:1603 read 0x8eeb66 thru 0x8f24b4 (0x394f bytes) 2024-09-13T09:17:55.036 INFO:teuthology.orchestra.run.smithi032.stdout:1604 write 0x8b512c8 thru 0x8b56b23 (0x585c bytes) 2024-09-13T09:17:55.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:17:55.041 INFO:teuthology.orchestra.run.smithi032.stdout:1605 read 0x40a4ee8 thru 0x40adf77 (0x9090 bytes) 2024-09-13T09:17:55.070 INFO:teuthology.orchestra.run.smithi032.stdout:1606 trunc from 0xe5cc7e2 to 0x3b235e4 2024-09-13T09:17:55.082 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:55.078+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:55.117 INFO:teuthology.orchestra.run.smithi032.stdout:1607 write 0x7ec8721 thru 0x7ed306e (0xa94e bytes) 2024-09-13T09:17:55.120 INFO:teuthology.orchestra.run.smithi032.stdout:1608 read 0x3827274 thru 0x3836cd8 (0xfa65 bytes) 2024-09-13T09:17:55.121 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:55.121 INFO:teuthology.orchestra.run.smithi032.stdout:1610 read 0x6b09740 thru 0x6b11a39 (0x82fa bytes) 2024-09-13T09:17:55.122 INFO:teuthology.orchestra.run.smithi032.stdout:1611 punch from 0x6b44f2b to 0x6b4d776, (0x884b bytes) 2024-09-13T09:17:55.126 INFO:teuthology.orchestra.run.smithi032.stdout:1612 trunc from 0x7ed306f to 0x6a08c77 2024-09-13T09:17:55.147 INFO:teuthology.orchestra.run.smithi032.stdout:1613 trunc from 0x6a08c77 to 0x8d0342f 2024-09-13T09:17:55.170 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:55.166+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:55.171 INFO:teuthology.orchestra.run.smithi032.stdout:1614 write 0x9505aa6 thru 0x950ce79 (0x73d4 bytes) 2024-09-13T09:17:55.187 INFO:teuthology.orchestra.run.smithi032.stdout:1616 writesame 0x96ae283 thru 0x96b0965 data_size 0xb(0x26e3 bytes) 2024-09-13T09:17:55.189 INFO:teuthology.orchestra.run.smithi032.stdout:1617 writesame 0x919466 thru 0x9241bb data_size 0xb(0xad56 bytes) 2024-09-13T09:17:55.199 INFO:teuthology.orchestra.run.smithi032.stdout:1619 write 0x43b8068 thru 0x43c4e3d (0xcdd6 bytes) 2024-09-13T09:17:55.203 INFO:teuthology.orchestra.run.smithi032.stdout:1620 read 0x4f9f9c8 thru 0x4fabc98 (0xc2d1 bytes) 2024-09-13T09:17:55.214 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:55.214 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:55.223 INFO:teuthology.orchestra.run.smithi032.stdout:1621 trunc from 0x96b0966 to 0x27f6988 2024-09-13T09:17:55.231 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:17:55.248 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:55.249 INFO:teuthology.orchestra.run.smithi032.stdout:1623 read 0x25d6333 thru 0x25da674 (0x4342 bytes) 2024-09-13T09:17:55.249 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:55.249 INFO:teuthology.orchestra.run.smithi032.stdout:1625 read 0xe91a4f thru 0xe93f71 (0x2523 bytes) 2024-09-13T09:17:55.250 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:55.250 INFO:teuthology.orchestra.run.smithi032.stdout:1627 read 0x9b9a45 thru 0x9c0aff (0x70bb bytes) 2024-09-13T09:17:55.251 INFO:teuthology.orchestra.run.smithi032.stdout:1628 read 0x1322a3c thru 0x1328c31 (0x61f6 bytes) 2024-09-13T09:17:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:1629 punch from 0xe46d83 to 0xe51d05, (0xaf82 bytes) 2024-09-13T09:17:55.252 INFO:teuthology.orchestra.run.smithi032.stdout:1630 clone 11 order 18 su 8192 sc 12 2024-09-13T09:17:55.271 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:55.268+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:55.373 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:55.372+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:55.398 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:55.398 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:55.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:17:55.474 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:55.472+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:55.540 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:55.540 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:55.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:17:55.574 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:55.572+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:55.675 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:55.670+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:55.675 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:55.675 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:55.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:17:55.776 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:55.776+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:55.858 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:55.858 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:55.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:17:55.881 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 7, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:17:55.881 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:17:55.881 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [2, 3, 4, 5, 6, 7, 0] 2024-09-13T09:17:55.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-09-13T09:17:55.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:17:55.918 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:55.914+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:55.977 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:55.970+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:55.996 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:55.997 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:56.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:17:56.041 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone10 from 0x27f6988 (overlap 0x541c9) to 0x10d28 2024-09-13T09:17:56.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:17:56.110 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:56.108+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:56.178 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:56.176+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:56.192 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:56.192 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:56.200 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-13T09:17:56.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:17:56.211 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:56.211 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:56.244 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:56.262 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:56.262 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:56.279 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:56.274+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:56.280 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:56.280 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:56.302 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:56.302 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:56.323 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:56.323 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:56.334 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:56.334 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:56.334 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:56.334 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:56.343 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:56.344 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:56.353 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:17:56.366 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-09-13T09:17:56.364+0000 7f1d86e9f640 -1 mon.a@0(leader).osd e45 definitely_dead 0 2024-09-13T09:17:56.367 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:56.367 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:56.389 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:56.384+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:56.481 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:56.480+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:56.486 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:56.486 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:56.486 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:56.486 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:56.486 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:56.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:17:56.567 INFO:teuthology.orchestra.run.smithi032.stdout:1631 write 0xc07c9fc thru 0xc086a8c (0xa091 bytes) 2024-09-13T09:17:56.582 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:56.580+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:56.583 INFO:teuthology.orchestra.run.smithi032.stdout:1632 punch from 0x558adf1 to 0x5594ddb, (0x9fea bytes) 2024-09-13T09:17:56.583 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:56.584 INFO:teuthology.orchestra.run.smithi032.stdout:1634 writesame 0x1d2d328 thru 0x1d3b0ed data_size 0x1(0xddc6 bytes) 2024-09-13T09:17:56.645 INFO:teuthology.orchestra.run.smithi032.stdout:1635 read 0x2df60b1 thru 0x2e02e39 (0xcd89 bytes) 2024-09-13T09:17:56.645 INFO:teuthology.orchestra.run.smithi032.stdout:1638 punch from 0x4d9c8 to 0x5ad68, (0xd3a0 bytes) 2024-09-13T09:17:56.645 INFO:teuthology.orchestra.run.smithi032.stdout:1639 read 0x3fe504f thru 0x3fec2c6 (0x7278 bytes) 2024-09-13T09:17:56.645 INFO:teuthology.orchestra.run.smithi032.stdout:1640 write 0x1466bfb thru 0x146897c (0x1d82 bytes) 2024-09-13T09:17:56.669 INFO:teuthology.orchestra.run.smithi032.stdout:1641 read 0xa45e9d5 thru 0xa469913 (0xaf3f bytes) 2024-09-13T09:17:56.669 INFO:teuthology.orchestra.run.smithi032.stdout:1642 punch from 0x1faef2c to 0x1fafc9f, (0xd73 bytes) 2024-09-13T09:17:56.669 INFO:teuthology.orchestra.run.smithi032.stdout:1643 write 0xa5b3659 thru 0xa5b4fcd (0x1975 bytes) 2024-09-13T09:17:56.676 INFO:teuthology.orchestra.run.smithi032.stdout:1644 write 0x5c8daf3 thru 0x5c93cb1 (0x61bf bytes) 2024-09-13T09:17:56.683 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:56.683 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:56.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:56.683 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:56.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.683 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3:ed699f0f:::rbd_data.118591b29fad.000000000000008c:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:17:56.683 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:17:56.684 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.821741615000001, 2024-09-13T09:17:56.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94878089700000001, 2024-09-13T09:17:56.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.684 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.684 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:17:56.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.684 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:17:56.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:56.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:17:56.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:17:56.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:17:56.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:17:56.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:17:56.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:17:56.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810654+0000", 2024-09-13T09:17:56.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94834837599999999 2024-09-13T09:17:56.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810755+0000", 2024-09-13T09:17:56.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100987 2024-09-13T09:17:56.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810828+0000", 2024-09-13T09:17:56.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2751999999999997e-05 2024-09-13T09:17:56.689 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.689 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.689 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.689 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4103 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.689 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973495+0000", 2024-09-13T09:17:56.690 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710293371000001, 2024-09-13T09:17:56.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023681364999999999, 2024-09-13T09:17:56.690 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.690 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.690 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.690 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.690 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.690 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4103 2024-09-13T09:17:56.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973495+0000", 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973495+0000", 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973495+0000", 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973497+0000", 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3190000000000002e-06 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973503+0000", 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0909999999999998e-06 2024-09-13T09:17:56.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973508+0000", 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.996995+0000", 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023486717000000001 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997080+0000", 2024-09-13T09:17:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5500999999999993e-05 2024-09-13T09:17:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997177+0000", 2024-09-13T09:17:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6325999999999999e-05 2024-09-13T09:17:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4104 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:17:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710263856999999, 2024-09-13T09:17:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023862547000000001, 2024-09-13T09:17:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4104 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:17:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973524+0000", 2024-09-13T09:17:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973527+0000", 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1200000000000002e-06 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973535+0000", 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2479999999999999e-06 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973548+0000", 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.365e-05 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997205+0000", 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023657088999999999 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997289+0000", 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3801000000000006e-05 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997387+0000", 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8317000000000001e-05 2024-09-13T09:17:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4105 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:17:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710235340000001, 2024-09-13T09:17:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024049968000000001, 2024-09-13T09:17:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4105 2024-09-13T09:17:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:17:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973556+0000", 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7420000000000002e-06 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973564+0000", 2024-09-13T09:17:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-13T09:17:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973569+0000", 2024-09-13T09:17:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-13T09:17:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997418+0000", 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023849052999999999 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997504+0000", 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6075999999999994e-05 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997603+0000", 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9455000000000002e-05 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.705 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4106 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.705 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:17:56.705 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710215749, 2024-09-13T09:17:56.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024364336, 2024-09-13T09:17:56.705 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.705 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.705 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.705 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.705 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.706 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4106 2024-09-13T09:17:56.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.706 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:17:56.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973575+0000", 2024-09-13T09:17:56.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.322e-06 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973594+0000", 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9031000000000001e-05 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973601+0000", 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5300000000000002e-06 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997648+0000", 2024-09-13T09:17:56.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024047731999999999 2024-09-13T09:17:56.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997748+0000", 2024-09-13T09:17:56.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9807000000000002e-05 2024-09-13T09:17:56.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997937+0000", 2024-09-13T09:17:56.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018924299999999999 2024-09-13T09:17:56.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.710 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4107 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.710 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:17:56.710 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710183304999999, 2024-09-13T09:17:56.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542690999999998, 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4107 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:17:56.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:17:56.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973614+0000", 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973621+0000", 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0110000000000003e-06 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973626+0000", 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0320000000000003e-06 2024-09-13T09:17:56.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997979+0000", 2024-09-13T09:17:56.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024353063000000001 2024-09-13T09:17:56.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998059+0000", 2024-09-13T09:17:56.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9221000000000006e-05 2024-09-13T09:17:56.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998148+0000", 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9506999999999995e-05 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4108 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710157697, 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024717223999999999, 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.715 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4108 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973633+0000", 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2089999999999999e-06 2024-09-13T09:17:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973639+0000", 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1469999999999998e-06 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973644+0000", 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5310000000000003e-06 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998175+0000", 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024531826999999999 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998257+0000", 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1668999999999993e-05 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998348+0000", 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1184e-05 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.719 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.720 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.720 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4109 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.720 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:17:56.720 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710138166, 2024-09-13T09:17:56.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024935052999999999, 2024-09-13T09:17:56.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.720 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.720 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.720 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.720 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4109 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973650+0000", 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973653+0000", 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.328e-06 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973659+0000", 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3269999999999997e-06 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973664+0000", 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8149999999999996e-06 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998375+0000", 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024711767999999999 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998484+0000", 2024-09-13T09:17:56.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108257 2024-09-13T09:17:56.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998586+0000", 2024-09-13T09:17:56.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101953 2024-09-13T09:17:56.724 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.724 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.724 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4110 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710120830999999, 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025120632, 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4110 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.725 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973678+0000", 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0712e-05 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973685+0000", 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9029999999999998e-06 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973690+0000", 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.955e-06 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998613+0000", 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024922546 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998695+0000", 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2066999999999999e-05 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998789+0000", 2024-09-13T09:17:56.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3813000000000005e-05 2024-09-13T09:17:56.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.729 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4111 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.729 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:17:56.729 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710093992999999, 2024-09-13T09:17:56.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025376396999999998, 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4111 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973694+0000", 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973697+0000", 2024-09-13T09:17:56.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.384e-06 2024-09-13T09:17:56.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973703+0000", 2024-09-13T09:17:56.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9970000000000004e-06 2024-09-13T09:17:56.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973707+0000", 2024-09-13T09:17:56.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7169999999999999e-06 2024-09-13T09:17:56.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998817+0000", 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025109446000000001 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998898+0000", 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1003999999999999e-05 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999071+0000", 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000173337 2024-09-13T09:17:56.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.734 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.734 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.734 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4112 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.734 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:17:56.734 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710077169, 2024-09-13T09:17:56.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025596280999999999, 2024-09-13T09:17:56.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.734 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.734 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.734 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4112 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:17:56.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973711+0000", 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973713+0000", 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2670000000000001e-06 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973719+0000", 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9379999999999997e-06 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973724+0000", 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6809999999999997e-06 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999115+0000", 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025390511000000001 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999202+0000", 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7787999999999994e-05 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999308+0000", 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105568 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4113 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 430080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710060024000001, 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025814090000000001, 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4113 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973728+0000", 2024-09-13T09:17:56.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973738+0000", 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4339999999999998e-06 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973745+0000", 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9639999999999997e-06 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973754+0000", 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4760000000000004e-06 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999360+0000", 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025606035999999999 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999443+0000", 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3393000000000006e-05 2024-09-13T09:17:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999543+0000", 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9351000000000005e-05 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4114 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710030168999999, 2024-09-13T09:17:56.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026036354000000001, 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4114 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973758+0000", 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973761+0000", 2024-09-13T09:17:56.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-13T09:17:56.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973767+0000", 2024-09-13T09:17:56.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6209999999999998e-06 2024-09-13T09:17:56.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973772+0000", 2024-09-13T09:17:56.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:17:56.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999593+0000", 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025820532 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999691+0000", 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8701000000000002e-05 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999795+0000", 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103467 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4115 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710012426, 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026260549000000001, 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.748 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4115 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973778+0000", 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.091e-06 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973784+0000", 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.268e-06 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973789+0000", 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4599999999999996e-06 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999836+0000", 2024-09-13T09:17:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026046980000000001 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999932+0000", 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5863e-05 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000037+0000", 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105264 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4116 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.709996038, 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026488986999999999, 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4116 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.753 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973792+0000", 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973794+0000", 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8950000000000001e-06 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973800+0000", 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9780000000000002e-06 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973811+0000", 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1192999999999999e-05 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000081+0000", 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026269942000000001 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000175+0000", 2024-09-13T09:17:56.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3819000000000004e-05 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000282+0000", 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106507 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4117 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:17:56.757 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.709972494000001, 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026706133999999999, 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4117 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:17:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973818+0000", 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6259999999999999e-06 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973825+0000", 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.737e-06 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973830+0000", 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4399999999999998e-06 2024-09-13T09:17:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000317+0000", 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026486948 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000398+0000", 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1160000000000002e-05 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000522+0000", 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012466800000000001 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4118 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.709954965, 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026891327, 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.762 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4118 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:17:56.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973833+0000", 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973836+0000", 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2350000000000002e-06 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973841+0000", 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7230000000000001e-06 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973846+0000", 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1829999999999996e-06 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000563+0000", 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026717797000000001 2024-09-13T09:17:56.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000630+0000", 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6152e-05 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000725+0000", 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5601999999999997e-05 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4119 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.709939027000001, 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027064775999999999, 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4119 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:17:56.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:17:56.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973849+0000", 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973852+0000", 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973859+0000", 2024-09-13T09:17:56.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7589999999999997e-06 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973864+0000", 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8119999999999998e-06 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000747+0000", 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026882653999999999 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000816+0000", 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8841e-05 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000915+0000", 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8933999999999998e-05 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14583 3.1f 3:feba6951:::rbd_data.11853ecd5f5f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:17:56.771 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:17:56.772 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.1497583509999991, 2024-09-13T09:17:56.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.050883949999999997, 2024-09-13T09:17:56.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.772 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:56.772 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.772 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.772 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.772 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14583 2024-09-13T09:17:56.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534028+0000", 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538304+0000", 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00427634 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538342+0000", 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7567000000000001e-05 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538357+0000", 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4936999999999999e-05 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538411+0000", 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4166000000000003e-05 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538539+0000", 2024-09-13T09:17:56.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000127966 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.539016+0000", 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477014 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.547890+0000", 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0088736430000000005 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584825+0000", 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036934966 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584866+0000", 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1514000000000003e-05 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584914+0000", 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8167000000000001e-05 2024-09-13T09:17:56.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16399 3.1f 3:fcbf421e:::rbd_data.1185f20bdb42.0000000000000019:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.655128+0000", 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0286606970000003, 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042135120999999998, 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:56.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16399 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655128+0000", 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655128+0000", 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:56.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655126+0000", 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655139+0000", 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3373000000000001e-05 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:56.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655164+0000", 2024-09-13T09:17:56.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4576999999999999e-05 2024-09-13T09:17:56.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:56.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655177+0000", 2024-09-13T09:17:56.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2853e-05 2024-09-13T09:17:56.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:56.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655257+0000", 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0445999999999995e-05 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655382+0000", 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012448900000000001 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.663115+0000", 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077335030000000001 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.697103+0000", 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033987512999999997 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.697165+0000", 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2731000000000006e-05 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.697223+0000", 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7651000000000001e-05 2024-09-13T09:17:56.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:56.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:56.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:56.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.697263+0000", 2024-09-13T09:17:56.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0061999999999997e-05 2024-09-13T09:17:56.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.784 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:56.784 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:56.784 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:56.785 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:56.678+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:56.786 INFO:teuthology.orchestra.run.smithi032.stdout:1647 read 0x42eb224 thru 0x42f95db (0xe3b8 bytes) 2024-09-13T09:17:56.786 INFO:teuthology.orchestra.run.smithi032.stdout:1648 punch from 0xa8bb0ba to 0xa8c28dd, (0x7823 bytes) 2024-09-13T09:17:56.786 INFO:teuthology.orchestra.run.smithi032.stdout:1649 punch from 0xda84e5 to 0xda8a27, (0x542 bytes) 2024-09-13T09:17:56.786 INFO:teuthology.orchestra.run.smithi032.stdout:1650 read 0x6ef0934 thru 0x6ef2860 (0x1f2d bytes) 2024-09-13T09:17:56.786 INFO:teuthology.orchestra.run.smithi032.stdout:1651 write 0x54ff384 thru 0x55061e7 (0x6e64 bytes) 2024-09-13T09:17:56.787 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:17:56.788 INFO:teuthology.orchestra.run.smithi032.stdout:1652 writesame 0x703b581 thru 0x7040335 data_size 0x3(0x4db5 bytes) 2024-09-13T09:17:56.788 INFO:teuthology.orchestra.run.smithi032.stdout:1653 read 0xb4ecc77 thru 0xb4faa71 (0xddfb bytes) 2024-09-13T09:17:56.789 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:56.789 INFO:teuthology.orchestra.run.smithi032.stdout:1655 read 0xb422991 thru 0xb42b080 (0x86f0 bytes) 2024-09-13T09:17:56.789 INFO:teuthology.orchestra.run.smithi032.stdout:1656 write 0x76b670 thru 0x77385f (0x81f0 bytes) 2024-09-13T09:17:56.789 INFO:teuthology.orchestra.run.smithi032.stdout:1658 read 0x7f2a61c thru 0x7f397e2 (0xf1c7 bytes) 2024-09-13T09:17:56.789 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:56.789 INFO:teuthology.orchestra.run.smithi032.stdout:1660 read 0x57b1775 thru 0x57b2e20 (0x16ac bytes) 2024-09-13T09:17:56.789 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:56.789 INFO:teuthology.orchestra.run.smithi032.stdout:1662 write 0xc93c55e thru 0xc948086 (0xbb29 bytes) 2024-09-13T09:17:56.789 INFO:teuthology.orchestra.run.smithi032.stdout:1663 read 0x69172f2 thru 0x6920496 (0x91a5 bytes) 2024-09-13T09:17:56.789 INFO:teuthology.orchestra.run.smithi032.stdout:1665 write 0x4e3df38 thru 0x4e4bcec (0xddb5 bytes) 2024-09-13T09:17:56.790 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:56.782+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:56.795 INFO:teuthology.orchestra.run.smithi032.stdout:1668 write 0x73310a2 thru 0x7337dde (0x6d3d bytes) 2024-09-13T09:17:56.804 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:56.804 INFO:teuthology.orchestra.run.smithi032.stdout:1670 read 0x8cbf910 thru 0x8ccb9a3 (0xc094 bytes) 2024-09-13T09:17:56.805 INFO:teuthology.orchestra.run.smithi032.stdout:1671 write 0x5ce6309 thru 0x5ce9c0a (0x3902 bytes) 2024-09-13T09:17:56.811 INFO:teuthology.orchestra.run.smithi032.stdout:1672 write 0x8f45ea1 thru 0x8f5052a (0xa68a bytes) 2024-09-13T09:17:56.828 INFO:teuthology.orchestra.run.smithi032.stdout:1674 read 0xbbce7d7 thru 0xbbd6eea (0x8714 bytes) 2024-09-13T09:17:56.828 INFO:teuthology.orchestra.run.smithi032.stdout:1675 read 0xc426372 thru 0xc431592 (0xb221 bytes) 2024-09-13T09:17:56.828 INFO:teuthology.orchestra.run.smithi032.stdout:1676 punch from 0x4c43bec to 0x4c51f01, (0xe315 bytes) 2024-09-13T09:17:56.829 INFO:teuthology.orchestra.run.smithi032.stdout:1677 write 0x48d3110 thru 0x48dc5b9 (0x94aa bytes) 2024-09-13T09:17:56.840 INFO:teuthology.orchestra.run.smithi032.stdout:1678 read 0x2f047ff thru 0x2f05a87 (0x1289 bytes) 2024-09-13T09:17:56.840 INFO:teuthology.orchestra.run.smithi032.stdout:1679 writesame 0x409ff42 thru 0x40a6835 data_size 0x6(0x68f4 bytes) 2024-09-13T09:17:56.868 INFO:teuthology.orchestra.run.smithi032.stdout:1680 trunc from 0xc948087 to 0x18458dd 2024-09-13T09:17:56.886 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:56.884+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:17:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:marked out osd.6. 2024-09-13T09:17:56.946 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:56.946 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:56.962 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:17:56.987 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:56.982+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:57.088 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:57.086+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:57.116 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:57.117 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:57.117 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:57.117 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:57.117 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:57.130 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:17:57.188 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:57.182+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:57.250 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:57.250 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:57.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5478 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:17:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710017322000001, 2024-09-13T09:17:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027749532, 2024-09-13T09:17:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5478 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539236+0000", 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.041e-06 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539242+0000", 2024-09-13T09:17:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0650000000000004e-06 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539246+0000", 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1500000000000001e-06 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566770+0000", 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027523178999999998 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566846+0000", 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5941999999999999e-05 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566984+0000", 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000138486 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5479 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710001058, 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027937460000000001, 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5479 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:17:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:17:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539250+0000", 2024-09-13T09:17:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539256+0000", 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539263+0000", 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3779999999999998e-06 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539273+0000", 2024-09-13T09:17:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0457e-05 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567010+0000", 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027736878 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567083+0000", 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3187000000000002e-05 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567188+0000", 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010491499999999999 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5480 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.70997384, 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028132641999999999, 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5480 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539280+0000", 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:17:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539297+0000", 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7147999999999999e-05 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539303+0000", 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9039999999999997e-06 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567230+0000", 2024-09-13T09:17:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027927250000000001 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567316+0000", 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5970999999999994e-05 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567411+0000", 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4523000000000003e-05 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5481 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.709943844, 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028317978000000001, 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5481 2024-09-13T09:17:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539307+0000", 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539311+0000", 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.264e-06 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539317+0000", 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:17:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539322+0000", 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6949999999999999e-06 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567446+0000", 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028124341000000001 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567527+0000", 2024-09-13T09:17:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0785000000000007e-05 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567626+0000", 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9103000000000003e-05 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5482 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:17:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.709926018999999, 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028619326, 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5482 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:17:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539325+0000", 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539333+0000", 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.055e-06 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539340+0000", 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4980000000000003e-06 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539344+0000", 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5369999999999999e-06 2024-09-13T09:17:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567655+0000", 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028310906 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567732+0000", 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6829999999999995e-05 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567945+0000", 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000212909 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5483 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.709902758, 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028784015999999999, 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5483 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539351+0000", 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539357+0000", 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0090000000000004e-06 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539361+0000", 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.357e-06 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567974+0000", 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028612385000000001 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568050+0000", 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5820000000000003e-05 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568133+0000", 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3551e-05 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5484 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.709886206, 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028949923999999998, 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5484 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539365+0000", 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539367+0000", 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539374+0000", 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539378+0000", 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568159+0000", 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028780490999999998 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568236+0000", 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7156000000000005e-05 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568316+0000", 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9598999999999995e-05 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5485 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.709066327, 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028332593, 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5485 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:17:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:17:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540189+0000", 2024-09-13T09:17:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540198+0000", 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9099999999999994e-06 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540206+0000", 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568360+0000", 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028154324000000001 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568436+0000", 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5821000000000005e-05 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568518+0000", 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1895000000000001e-05 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5486 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.709013318, 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028479536999999999, 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5486 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540241+0000", 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4290000000000001e-06 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540249+0000", 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9480000000000008e-06 2024-09-13T09:17:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540255+0000", 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6019999999999999e-06 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568543+0000", 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028288260999999999 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568636+0000", 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2627999999999995e-05 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568718+0000", 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2213999999999996e-05 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5487 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:17:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.708992233, 2024-09-13T09:17:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028653435000000001, 2024-09-13T09:17:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5487 2024-09-13T09:17:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540259+0000", 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540262+0000", 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.413e-06 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540268+0000", 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3940000000000001e-06 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540272+0000", 2024-09-13T09:17:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3989999999999997e-06 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568745+0000", 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028472173999999999 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568822+0000", 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7694000000000005e-05 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568913+0000", 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0779000000000006e-05 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5488 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 430080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.708974832999999, 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028788326999999999, 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5488 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:17:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:17:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540279+0000", 2024-09-13T09:17:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2409999999999998e-06 2024-09-13T09:17:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540285+0000", 2024-09-13T09:17:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3180000000000003e-06 2024-09-13T09:17:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540300+0000", 2024-09-13T09:17:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4625e-05 2024-09-13T09:17:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568939+0000", 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028639688 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569016+0000", 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6556000000000004e-05 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569065+0000", 2024-09-13T09:17:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9246000000000003e-05 2024-09-13T09:17:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5489 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:17:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.708947239, 2024-09-13T09:17:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028875332, 2024-09-13T09:17:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5489 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540304+0000", 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540307+0000", 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4370000000000001e-06 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540313+0000", 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540318+0000", 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569079+0000", 2024-09-13T09:17:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028760714 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569128+0000", 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8702999999999998e-05 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569180+0000", 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2105999999999999e-05 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5490 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:17:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.708928744, 2024-09-13T09:17:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029041027000000001, 2024-09-13T09:17:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5490 2024-09-13T09:17:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:17:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540326+0000", 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9840000000000001e-06 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540332+0000", 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9730000000000002e-06 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540338+0000", 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3550000000000001e-06 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569202+0000", 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028863521 2024-09-13T09:17:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569276+0000", 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569364+0000", 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7594000000000002e-05 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5491 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:17:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.70890936, 2024-09-13T09:17:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029221864, 2024-09-13T09:17:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5491 2024-09-13T09:17:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540350+0000", 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4250000000000001e-06 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540356+0000", 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2960000000000004e-06 2024-09-13T09:17:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540361+0000", 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569388+0000", 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029026744 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569474+0000", 2024-09-13T09:17:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6519000000000003e-05 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569564+0000", 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9969999999999994e-05 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5492 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:17:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.708886063, 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029401203000000001, 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5492 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:17:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540365+0000", 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540368+0000", 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540374+0000", 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9349999999999999e-06 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540378+0000", 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.651e-06 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569591+0000", 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029212557 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569668+0000", 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569767+0000", 2024-09-13T09:17:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9061000000000006e-05 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5493 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.708869199, 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029599424999999999, 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5493 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540382+0000", 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540384+0000", 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0430000000000002e-06 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540390+0000", 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540395+0000", 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5990000000000004e-06 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569795+0000", 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029400582000000001 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569880+0000", 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5210999999999994e-05 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569982+0000", 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101584 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5494 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.708853114, 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029770637999999999, 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5494 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540398+0000", 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540406+0000", 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540413+0000", 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540418+0000", 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8289999999999998e-06 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570007+0000", 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029588851999999999 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570084+0000", 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7253e-05 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570169+0000", 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5391999999999998e-05 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5495 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.708814798000001, 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029938664, 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5495 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540440+0000", 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0319999999999999e-06 2024-09-13T09:17:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540446+0000", 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8549999999999996e-06 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540451+0000", 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570198+0000", 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029746060000000001 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570284+0000", 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6044999999999994e-05 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570376+0000", 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2071000000000007e-05 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5496 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.708795843000001, 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030121617999999999, 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5496 2024-09-13T09:17:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540458+0000", 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540464+0000", 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.156e-06 2024-09-13T09:17:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540468+0000", 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570404+0000", 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029935632 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570488+0000", 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3873000000000001e-05 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570578+0000", 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9757000000000001e-05 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5497 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:17:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.708779279, 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030305259000000001, 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5497 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:17:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540472+0000", 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540474+0000", 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0490000000000002e-06 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540480+0000", 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1970000000000001e-06 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540491+0000", 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0964e-05 2024-09-13T09:17:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570608+0000", 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030116105000000001 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570688+0000", 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0143999999999997e-05 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570778+0000", 2024-09-13T09:17:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0131999999999997e-05 2024-09-13T09:17:57.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.344 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:57.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:17:57.346 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:57.344+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:57.447 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:57.444+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:57.521 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:57.521 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:57.521 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:57.522 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:57.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:17:57.578 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:57.576+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:57.648 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:17:57.644+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:17:57.702 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:57.702 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:57.702 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:57.702 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:57.702 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:57.716 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:17:57.748 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:57.746+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:57.849 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:57.848+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:57.890 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:57.890 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:57.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:57.890 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:57.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.890 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3022 3.9 e31/22 3:91681905:::rbd_data.118591b29fad.0000000000000209:head v 31'171, mlcod=31'169)", 2024-09-13T09:17:57.890 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:17:57.890 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.420152430000002, 2024-09-13T09:17:57.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002956209, 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468237+0000", 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468300+0000", 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3202999999999998e-05 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468353+0000", 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2735999999999998e-05 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468374+0000", 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0930000000000001e-05 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468568+0000", 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000194262 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468627+0000", 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8743000000000003e-05 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.471132+0000", 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025055540000000001 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.471196+0000", 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3590999999999995e-05 2024-09-13T09:17:57.894 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "pg_update_log_missing(3.4 epoch 35/22 rep_tid 113 entries 35'31 (0'0) error 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head by client.4485.0:3311 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 31'29)", 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.360471917, 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033204409999999999, 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527918+0000", 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527943+0000", 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4752999999999998e-05 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527988+0000", 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4693000000000001e-05 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528029+0000", 2024-09-13T09:17:57.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1155999999999997e-05 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528194+0000", 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164792 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.531241+0000", 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003046867 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3323 3.4 e35/22 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head v 35'34, mlcod=35'31)", 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.352596991, 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037621640000000001, 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535793+0000", 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535811+0000", 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8170000000000001e-05 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535840+0000", 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9105999999999999e-05 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535877+0000", 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7017000000000002e-05 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536001+0000", 2024-09-13T09:17:57.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012342899999999999 2024-09-13T09:17:57.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536073+0000", 2024-09-13T09:17:57.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2123e-05 2024-09-13T09:17:57.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:57.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539433+0000", 2024-09-13T09:17:57.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033596630000000001 2024-09-13T09:17:57.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539557+0000", 2024-09-13T09:17:57.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012470600000000001 2024-09-13T09:17:57.903 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.903 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.903 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:15977 3.4 3:27c24f46:::rbd_data.11856af9a4d3.0000000000000005:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8933274779999998, 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003633163, 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15977 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:17:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:17:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995063+0000", 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995071+0000", 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3930000000000002e-06 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995086+0000", 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4970999999999999e-05 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995095+0000", 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8810000000000002e-06 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995180+0000", 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5012000000000005e-05 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995378+0000", 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019788600000000001 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.998698+0000", 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003320267 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16010 3.4 3:27c24f46:::rbd_data.11856af9a4d3.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8877186860000004, 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037539579999999999, 2024-09-13T09:17:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16010 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000672+0000", 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000679+0000", 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9120000000000001e-06 2024-09-13T09:17:57.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000693+0000", 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3745e-05 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000702+0000", 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.064e-06 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000763+0000", 2024-09-13T09:17:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1309000000000005e-05 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000877+0000", 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114214 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.001145+0000", 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000267765 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.002514+0000", 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013688730000000001 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004301+0000", 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017869839999999999 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004380+0000", 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8861000000000003e-05 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004427+0000", 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7713000000000002e-05 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16047 3.0 3:002d813e:::rbd_data.11856af9a4d3.0000000000000014:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8788416840000002, 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030667929999999999, 2024-09-13T09:17:57.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16047 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009548+0000", 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009559+0000", 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0936e-05 2024-09-13T09:17:57.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009579+0000", 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9976e-05 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009590+0000", 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1228e-05 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009670+0000", 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9765999999999995e-05 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009776+0000", 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106049 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009991+0000", 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021494999999999999 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.011621+0000", 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016294650000000001 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012469+0000", 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000848598 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012551+0000", 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2144999999999994e-05 2024-09-13T09:17:57.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012617+0000", 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5827999999999994e-05 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16226 3.4 3:22419c10:::rbd_data.1185f20bdb42.0000000000000013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.6598415800000001, 2024-09-13T09:17:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033492510000000001, 2024-09-13T09:17:57.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.921 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:57.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.921 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16226 2024-09-13T09:17:57.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.921 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:17:57.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228546+0000", 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999957 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228569+0000", 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2347000000000001e-05 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228618+0000", 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9509999999999999e-05 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228643+0000", 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4759000000000001e-05 2024-09-13T09:17:57.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228738+0000", 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5273999999999996e-05 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228819+0000", 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0623000000000004e-05 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228921+0000", 2024-09-13T09:17:57.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010215199999999999 2024-09-13T09:17:57.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:57.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.230000+0000", 2024-09-13T09:17:57.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010793510000000001 2024-09-13T09:17:57.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:57.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231718+0000", 2024-09-13T09:17:57.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001717873 2024-09-13T09:17:57.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231813+0000", 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4473000000000002e-05 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231900+0000", 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7158999999999997e-05 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16293 3.4 3:22419c10:::rbd_data.1185f20bdb42.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.5062308990000002, 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015655326000000001, 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16293 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382158+0000", 2024-09-13T09:17:57.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382176+0000", 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7863000000000002e-05 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382211+0000", 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.481e-05 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382226+0000", 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.537e-05 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382354+0000", 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012803599999999999 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382591+0000", 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023629099999999999 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.385269+0000", 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002678532 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397515+0000", 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012245946000000001 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397551+0000", 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5856999999999999e-05 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397699+0000", 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148255 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397817+0000", 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117299 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4485.0:16293 3.4 e42/41 ondisk, result = 0)", 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.495899359, 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0050343319999999999, 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392491+0000", 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392500+0000", 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8300000000000002e-06 2024-09-13T09:17:57.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392529+0000", 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9260000000000001e-05 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392542+0000", 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2969999999999999e-05 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397468+0000", 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0049256880000000001 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397496+0000", 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8209999999999999e-05 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397527+0000", 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1276999999999999e-05 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16837 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [call rbd.sparse_copyup in=173207b,set-alloc-hint object_size 16777216 write_size 16777216,write 3145728~10909 in=10909b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6282685460000001, 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004073887, 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16837 2024-09-13T09:17:57.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260121+0000", 2024-09-13T09:17:57.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:57.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260266+0000", 2024-09-13T09:17:57.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014495999999999999 2024-09-13T09:17:57.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260287+0000", 2024-09-13T09:17:57.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1336e-05 2024-09-13T09:17:57.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260296+0000", 2024-09-13T09:17:57.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6170000000000007e-06 2024-09-13T09:17:57.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260352+0000", 2024-09-13T09:17:57.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6252999999999998e-05 2024-09-13T09:17:57.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260520+0000", 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000168381 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260807+0000", 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028624100000000001 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.262062+0000", 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012551490000000001 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264128+0000", 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020660380000000001 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264167+0000", 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9714e-05 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264198+0000", 2024-09-13T09:17:57.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0105000000000001e-05 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16964 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [stat,zero 8912896~6815744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.358046+0000", 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.5303459899999998, 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027898210000000001, 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16964 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358046+0000", 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358046+0000", 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358044+0000", 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358055+0000", 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0958000000000001e-05 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358081+0000", 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5978000000000001e-05 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358092+0000", 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.116e-05 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358184+0000", 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2243000000000004e-05 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358320+0000", 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013623500000000001 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.360836+0000", 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025156699999999998 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:17005 3.1a e43/41 3:5e94b56b:::rbd_data.11859d3d11dd.0000000000000008:head v 43'54, mlcod=42'52)", 2024-09-13T09:17:57.947 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.5027038890000002, 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031754880000000002, 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:17:57.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:17:57.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:57.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.386997+0000", 2024-09-13T09:17:57.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00130933 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387013+0000", 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5752000000000001e-05 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387022+0000", 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1689999999999998e-06 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387092+0000", 2024-09-13T09:17:57.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9385000000000001e-05 2024-09-13T09:17:57.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387127+0000", 2024-09-13T09:17:57.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5061000000000002e-05 2024-09-13T09:17:57.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:57.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.388821+0000", 2024-09-13T09:17:57.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001694463 2024-09-13T09:17:57.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.388864+0000", 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2721e-05 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:17028 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [stat,zero 4194304~4718592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.4788854359999997, 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036807770000000001, 2024-09-13T09:17:57.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.953 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.953 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 17028 2024-09-13T09:17:57.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.953 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:17:57.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409504+0000", 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409515+0000", 2024-09-13T09:17:57.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0485e-05 2024-09-13T09:17:57.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409540+0000", 2024-09-13T09:17:57.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4808e-05 2024-09-13T09:17:57.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409550+0000", 2024-09-13T09:17:57.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0376e-05 2024-09-13T09:17:57.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409616+0000", 2024-09-13T09:17:57.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6130999999999994e-05 2024-09-13T09:17:57.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409725+0000", 2024-09-13T09:17:57.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108614 2024-09-13T09:17:57.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.413188+0000", 2024-09-13T09:17:57.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003462849 2024-09-13T09:17:57.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.957 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.960 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18757 3.4 3:21c7a0ac:::rbd_data.1185b72576e4.0000000000000074:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.258357744, 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066789190000000002, 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18757 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630032+0000", 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630044+0000", 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2245999999999999e-05 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630067+0000", 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3404000000000002e-05 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630078+0000", 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1158999999999999e-05 2024-09-13T09:17:57.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630156+0000", 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7520000000000003e-05 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630357+0000", 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020064700000000001 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:57.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.631878+0000", 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015212940000000001 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636574+0000", 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046960719999999999 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636612+0000", 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7499999999999997e-05 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636669+0000", 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7286e-05 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636713+0000", 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4530000000000002e-05 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18820 3.4 3:250382e8:::rbd_data.1185c2de398f.0000000000000005:e [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.648443+0000", 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.23994944, 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027588159999999999, 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18820 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648443+0000", 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648443+0000", 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648443+0000", 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-13T09:17:57.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648444+0000", 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.592e-06 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648447+0000", 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648450+0000", 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7259999999999998e-06 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651052+0000", 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0026020259999999999 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651107+0000", 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5074999999999997e-05 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651202+0000", 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4293e-05 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18821 3.4 3:250382e8:::rbd_data.1185c2de398f.0000000000000005:e [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.648454+0000", 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.2399381, 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002981521, 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18821 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648454+0000", 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648454+0000", 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648454+0000", 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648455+0000", 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.066e-06 2024-09-13T09:17:57.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648458+0000", 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3340000000000002e-06 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648461+0000", 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651241+0000", 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002779625 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651336+0000", 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4946999999999997e-05 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651436+0000", 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9925000000000003e-05 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18822 3.4 3:250382e8:::rbd_data.1185c2de398f.0000000000000005:e [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:17:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:17:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.239928659, 2024-09-13T09:17:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031435460000000001, 2024-09-13T09:17:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18822 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648463+0000", 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:17:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-06 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648468+0000", 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1999999999999999e-06 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648470+0000", 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7389999999999999e-06 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651484+0000", 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003014128 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651541+0000", 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6304999999999997e-05 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651607+0000", 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6280999999999998e-05 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18951 3.4 3:2525adb5:::rbd_data.1185b72576e4.0000000000000139:head [set-alloc-hint object_size 262144 write_size 262144,write 24576~7405] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.783753+0000", 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.104639138, 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027657210000000001, 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18951 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.783753+0000", 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.783753+0000", 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.783751+0000", 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.783771+0000", 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9806e-05 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.783800+0000", 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9105999999999999e-05 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.783817+0000", 2024-09-13T09:17:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7473e-05 2024-09-13T09:17:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.783958+0000", 2024-09-13T09:17:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014059899999999999 2024-09-13T09:17:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.784181+0000", 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000223682 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.784416+0000", 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023429500000000001 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.785496+0000", 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010806769999999999 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.786464+0000", 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00096759700000000005 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.786497+0000", 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.307e-05 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.786519+0000", 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1732000000000001e-05 2024-09-13T09:17:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19013 3.4 3:21c7a0ac:::rbd_data.1185b72576e4.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.874194+0000", 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.0141982, 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033037850000000001, 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19013 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874194+0000", 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874194+0000", 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874191+0000", 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874207+0000", 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6441000000000001e-05 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874235+0000", 2024-09-13T09:17:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7554000000000001e-05 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874246+0000", 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1114999999999999e-05 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874331+0000", 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4608e-05 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874450+0000", 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011943900000000001 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874607+0000", 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015705600000000001 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.875492+0000", 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00088451199999999995 2024-09-13T09:17:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.877353+0000", 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018616220000000001 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.877437+0000", 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3698999999999999e-05 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.877498+0000", 2024-09-13T09:17:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0745999999999997e-05 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19037 3.4 3:277a4fb8:::rbd_data.1185b72576e4.000000000000015a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.883824+0000", 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.004567891, 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029514070000000001, 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19037 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883824+0000", 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883824+0000", 2024-09-13T09:17:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883822+0000", 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883836+0000", 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3910000000000001e-05 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883891+0000", 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4639999999999999e-05 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883912+0000", 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1043000000000002e-05 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.884074+0000", 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000162039 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.884308+0000", 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023380800000000001 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.884623+0000", 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031597400000000001 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.886279+0000", 2024-09-13T09:17:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001655091 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.886674+0000", 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039576299999999999 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.886733+0000", 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8390000000000002e-05 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.886776+0000", 2024-09-13T09:17:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2969000000000001e-05 2024-09-13T09:17:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:57.999 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:58.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:17:58.001 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:57.994+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:58.102 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:58.098+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:58.172 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:58.172 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:17:58.172 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:58.186 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:17:58.234 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:17:58.232+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:17:58.304 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:58.300+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:58.313 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:58.313 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:17:58.313 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:17:58.313 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:17:58.313 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:58.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:17:58.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:17:58.414 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:17:58.412+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:17:58.503 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:17:58.503 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:17:58.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:17:58.504 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:17:58.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.504 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:17:58.504 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:17:58.504 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.966895653, 2024-09-13T09:17:58.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:17:58.504 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.504 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:17:58.504 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.504 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.505 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.505 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:17:58.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.505 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:17:58.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:17:58.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:17:58.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:17:58.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:17:58.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:17:58.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:17:58.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:17:58.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:17:58.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:17:58.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:17:58.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:17:58.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:17:58.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:17:58.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:17:58.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:17:58.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:17:58.510 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.510 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.510 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19325 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2514944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5734189870000002, 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041428072000000003, 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19325 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.511 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:17:58.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:17:58.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:17:58.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929839+0000", 2024-09-13T09:17:58.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:17:58.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929845+0000", 2024-09-13T09:17:58.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9510000000000003e-06 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929850+0000", 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971169+0000", 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041318743999999998 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971214+0000", 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971265+0000", 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0963e-05 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19326 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2613248~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:17:58.515 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.573402051, 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041534459000000003, 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19326 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:17:58.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:17:58.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:17:58.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929856+0000", 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0470000000000001e-06 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929862+0000", 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3300000000000004e-06 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929867+0000", 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:17:58.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971293+0000", 2024-09-13T09:17:58.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041425785999999999 2024-09-13T09:17:58.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971337+0000", 2024-09-13T09:17:58.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4240999999999999e-05 2024-09-13T09:17:58.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971389+0000", 2024-09-13T09:17:58.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1483999999999997e-05 2024-09-13T09:17:58.521 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.521 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.521 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.521 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19327 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2711552~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.522 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:17:58.522 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5733849850000001, 2024-09-13T09:17:58.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041642875000000003, 2024-09-13T09:17:58.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.522 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.522 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.522 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.523 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.523 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19327 2024-09-13T09:17:58.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.523 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:17:58.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:17:58.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:17:58.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929873+0000", 2024-09-13T09:17:58.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.26e-06 2024-09-13T09:17:58.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929879+0000", 2024-09-13T09:17:58.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0519999999999998e-06 2024-09-13T09:17:58.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929884+0000", 2024-09-13T09:17:58.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:17:58.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971416+0000", 2024-09-13T09:17:58.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041532122999999997 2024-09-13T09:17:58.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971460+0000", 2024-09-13T09:17:58.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4456999999999998e-05 2024-09-13T09:17:58.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971514+0000", 2024-09-13T09:17:58.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3579000000000003e-05 2024-09-13T09:17:58.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.529 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.529 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19328 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2809856~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.530 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:17:58.530 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.573368222, 2024-09-13T09:17:58.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041750662000000001, 2024-09-13T09:17:58.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.530 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.530 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.530 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.530 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19328 2024-09-13T09:17:58.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.530 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:17:58.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929890+0000", 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929896+0000", 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929901+0000", 2024-09-13T09:17:58.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:17:58.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971542+0000", 2024-09-13T09:17:58.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041640442 2024-09-13T09:17:58.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971587+0000", 2024-09-13T09:17:58.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4972999999999998e-05 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971639+0000", 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1990000000000002e-05 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19329 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2908160~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:17:58.534 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5733262300000002, 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041833958999999997, 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19329 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929929+0000", 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929932+0000", 2024-09-13T09:17:58.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0699999999999998e-06 2024-09-13T09:17:58.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929941+0000", 2024-09-13T09:17:58.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.149e-06 2024-09-13T09:17:58.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929947+0000", 2024-09-13T09:17:58.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.699e-06 2024-09-13T09:17:58.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971666+0000", 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041719224999999999 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971711+0000", 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4823000000000001e-05 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971764+0000", 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2741999999999997e-05 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19330 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 3006464~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5733034890000002, 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041949980999999997, 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19330 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929952+0000", 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929955+0000", 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929961+0000", 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1199999999999999e-06 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929966+0000", 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971792+0000", 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041826680999999998 2024-09-13T09:17:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971838+0000", 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5939000000000002e-05 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971903+0000", 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4499999999999996e-05 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19331 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 3104768~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5732865220000001, 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042058848000000003, 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19331 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929969+0000", 2024-09-13T09:17:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929975+0000", 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.434e-06 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929981+0000", 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929986+0000", 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1780000000000002e-06 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971931+0000", 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041944417999999997 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971976+0000", 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4962000000000001e-05 2024-09-13T09:17:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972029+0000", 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2768e-05 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19332 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 65536~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.573259352, 2024-09-13T09:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042158568, 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19332 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:17:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929996+0000", 2024-09-13T09:17:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929999+0000", 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.306e-06 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930005+0000", 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0379999999999996e-06 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930010+0000", 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0969999999999996e-06 2024-09-13T09:17:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972057+0000", 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042047117000000002 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972102+0000", 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5422e-05 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972155+0000", 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2964999999999999e-05 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19333 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 163840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5732422619999999, 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042266805999999997, 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19333 2024-09-13T09:17:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:17:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930016+0000", 2024-09-13T09:17:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930022+0000", 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2550000000000003e-06 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930027+0000", 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8060000000000002e-06 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972183+0000", 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042156276999999999 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972228+0000", 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5317e-05 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972281+0000", 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2398999999999998e-05 2024-09-13T09:17:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19334 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 262144~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.573224417, 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042375573, 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19334 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930031+0000", 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930034+0000", 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930040+0000", 2024-09-13T09:17:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0789999999999997e-06 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930045+0000", 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0139999999999998e-06 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972307+0000", 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042262685000000001 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972352+0000", 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5021000000000001e-05 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972407+0000", 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4945000000000003e-05 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19335 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 360448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.573207603, 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042483785000000003, 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19335 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930048+0000", 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930050+0000", 2024-09-13T09:17:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.136e-06 2024-09-13T09:17:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930056+0000", 2024-09-13T09:17:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:17:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930061+0000", 2024-09-13T09:17:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9289999999999997e-06 2024-09-13T09:17:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972435+0000", 2024-09-13T09:17:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042373347999999998 2024-09-13T09:17:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972479+0000", 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4597e-05 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972532+0000", 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3034000000000001e-05 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19336 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 458752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:17:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5731906659999999, 2024-09-13T09:17:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042590488000000003, 2024-09-13T09:17:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19336 2024-09-13T09:17:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930065+0000", 2024-09-13T09:17:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930067+0000", 2024-09-13T09:17:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2110000000000001e-06 2024-09-13T09:17:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930074+0000", 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4640000000000003e-06 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930079+0000", 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7709999999999997e-06 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972560+0000", 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042481738999999998 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972604+0000", 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4060999999999997e-05 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972656+0000", 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.164e-05 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19337 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5731734909999999, 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042697997000000001, 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19337 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930082+0000", 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930085+0000", 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.221e-06 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930091+0000", 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.942e-06 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930096+0000", 2024-09-13T09:17:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.04e-06 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972683+0000", 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042587885999999998 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972728+0000", 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4058999999999999e-05 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972781+0000", 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3156999999999999e-05 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19342 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1048576~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.571256064, 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041418260999999998, 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19342 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932002+0000", 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3180000000000001e-06 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932008+0000", 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2199999999999997e-06 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932013+0000", 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973319+0000", 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041305738000000002 2024-09-13T09:17:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973364+0000", 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5809000000000001e-05 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973418+0000", 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3888000000000003e-05 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19343 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1146880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5712288569999999, 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041515107000000002, 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19343 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:17:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932034+0000", 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932040+0000", 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2260000000000002e-06 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932045+0000", 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5589999999999999e-06 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973446+0000", 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041401070999999998 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973491+0000", 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5021999999999997e-05 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973542+0000", 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1465000000000003e-05 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19344 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.571206331, 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041617848999999998, 2024-09-13T09:17:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19344 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932049+0000", 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932052+0000", 2024-09-13T09:17:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4540000000000001e-06 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932064+0000", 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1911e-05 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932079+0000", 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4765e-05 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973570+0000", 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041491800000000002 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973614+0000", 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4149e-05 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973668+0000", 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3198999999999997e-05 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19345 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1343488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5711701599999999, 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041707699000000001, 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19345 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932085+0000", 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932089+0000", 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7009999999999998e-06 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932097+0000", 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3010000000000007e-06 2024-09-13T09:17:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932104+0000", 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9260000000000003e-06 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973695+0000", 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041591056000000001 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973741+0000", 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6003e-05 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973794+0000", 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2265999999999997e-05 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19346 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1441792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:17:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.5711460590000002, 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041808933999999999, 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19346 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:17:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932112+0000", 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932119+0000", 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8369999999999999e-06 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932125+0000", 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973822+0000", 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041697369999999997 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973867+0000", 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4648999999999999e-05 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973919+0000", 2024-09-13T09:17:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2451000000000003e-05 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19347 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1540096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.571126606, 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041915045999999997, 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:17:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19347 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932129+0000", 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932131+0000", 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.278e-06 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932139+0000", 2024-09-13T09:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.43e-06 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932145+0000", 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7939999999999999e-06 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973946+0000", 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041801724999999998 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973993+0000", 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6245000000000002e-05 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.974045+0000", 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1950999999999998e-05 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:17:58.606 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:17:58.608 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:17:58.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:17:58.609 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:17:58.609 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:17:58.609 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:17:58.610 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:17:58.610 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:17:58.610 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:17:58.610 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:17:58.611 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:58.604+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:58.624 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:17:58.624 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:17:58.643 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:17:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:17:58.661 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:17:58.661 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:17:58.679 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:17:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:17:58.709 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:58.702+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:58.747 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:58.748 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:58.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:17:58.850 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:58.846+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:58.914 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:58.910+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:58.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:17:58.940 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:58.940 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:58.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:17:58.958 INFO:teuthology.orchestra.run.smithi032.stdout:1681 trunc from 0x18458dd to 0x7c56218 2024-09-13T09:17:58.984 INFO:teuthology.orchestra.run.smithi032.stdout:1682 write 0x9cbb710 thru 0x9cc39ac (0x829d bytes) 2024-09-13T09:17:58.996 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:58.996 INFO:teuthology.orchestra.run.smithi032.stdout:1684 write 0x3248c07 thru 0x324e2bd (0x56b7 bytes) 2024-09-13T09:17:59.008 INFO:teuthology.orchestra.run.smithi032.stdout:1685 read 0x2b6320b thru 0x2b67da4 (0x4b9a bytes) 2024-09-13T09:17:59.008 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:59.010 INFO:teuthology.orchestra.run.smithi032.stdout:1687 trunc from 0x9cc39ad to 0x8a83fae 2024-09-13T09:17:59.014 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:59.012+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:59.033 INFO:teuthology.orchestra.run.smithi032.stdout:1688 trunc from 0x8a83fae to 0x2647200 2024-09-13T09:17:59.069 INFO:teuthology.orchestra.run.smithi032.stdout:1689 write 0xaf38a8c thru 0xaf4776e (0xece3 bytes) 2024-09-13T09:17:59.085 INFO:teuthology.orchestra.run.smithi032.stdout:1690 punch from 0x9c3cbd0 to 0x9c4996a, (0xcd9a bytes) 2024-09-13T09:17:59.085 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:59.085 INFO:teuthology.orchestra.run.smithi032.stdout:1692 read 0x590b97a thru 0x5917205 (0xb88c bytes) 2024-09-13T09:17:59.085 INFO:teuthology.orchestra.run.smithi032.stdout:1693 punch from 0x3d8738d to 0x3d93a84, (0xc6f7 bytes) 2024-09-13T09:17:59.085 INFO:teuthology.orchestra.run.smithi032.stdout:1694 write 0x6bd382d thru 0x6bd9ce3 (0x64b7 bytes) 2024-09-13T09:17:59.093 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:59.093 INFO:teuthology.orchestra.run.smithi032.stdout:1696 write 0x2aa3dd6 thru 0x2ab06c0 (0xc8eb bytes) 2024-09-13T09:17:59.104 INFO:teuthology.orchestra.run.smithi032.stdout:1697 write 0x6b2e657 thru 0x6b326ce (0x4078 bytes) 2024-09-13T09:17:59.113 INFO:teuthology.orchestra.run.smithi032.stdout:1699 write 0xde33d1 thru 0xdf1a08 (0xe638 bytes) 2024-09-13T09:17:59.115 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:59.110+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:59.134 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:59.134 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:59.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:17:59.242 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:59.238+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:59.318 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:17:59.314+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:17:59.320 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:59.320 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:59.334 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:17:59.401 INFO:teuthology.orchestra.run.smithi032.stdout:1701 read 0x5a866bb thru 0x5a93b81 (0xd4c7 bytes) 2024-09-13T09:17:59.401 INFO:teuthology.orchestra.run.smithi032.stdout:1702 punch from 0x156c0a3 to 0x15736e4, (0x7641 bytes) 2024-09-13T09:17:59.401 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:59.415 INFO:teuthology.orchestra.run.smithi032.stdout:1704 write 0xc8c82cd thru 0xc8d6dc1 (0xeaf5 bytes) 2024-09-13T09:17:59.421 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:59.418+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:59.431 INFO:teuthology.orchestra.run.smithi032.stdout:1705 write 0xbb76ddf thru 0xbb7ddfa (0x701c bytes) 2024-09-13T09:17:59.441 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:59.459 INFO:teuthology.orchestra.run.smithi032.stdout:1709 write 0xe649d8c thru 0xe64d98e (0x3c03 bytes) 2024-09-13T09:17:59.466 INFO:teuthology.orchestra.run.smithi032.stdout:1710 writesame 0xe50da98 thru 0xe51952c data_size 0x29(0xba95 bytes) 2024-09-13T09:17:59.476 INFO:teuthology.orchestra.run.smithi032.stdout:1711 punch from 0x875aaae to 0x876a3ab, (0xf8fd bytes) 2024-09-13T09:17:59.476 INFO:teuthology.orchestra.run.smithi032.stdout:1713 writesame 0x6b6ee06 thru 0x6b7274c data_size 0x1f(0x3947 bytes) 2024-09-13T09:17:59.482 INFO:teuthology.orchestra.run.smithi032.stdout:1714 write 0x17c1a6 thru 0x18a0ed (0xdf48 bytes) 2024-09-13T09:17:59.520 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:17:59.514+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:17:59.537 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:59.537 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:59.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:17:59.589 INFO:teuthology.orchestra.run.smithi032.stdout:1715 read 0xd5decf6 thru 0xd5eaf9e (0xc2a9 bytes) 2024-09-13T09:17:59.589 INFO:teuthology.orchestra.run.smithi032.stdout:1716 write 0x4cecf78 thru 0x4cfb529 (0xe5b2 bytes) 2024-09-13T09:17:59.607 INFO:teuthology.orchestra.run.smithi032.stdout:1719 write 0x6b21b83 thru 0x6b2ca04 (0xae82 bytes) 2024-09-13T09:17:59.614 INFO:teuthology.orchestra.run.smithi032.stdout:1723 write 0x47defa3 thru 0x47ec0a7 (0xd105 bytes) 2024-09-13T09:17:59.627 INFO:teuthology.orchestra.run.smithi032.stdout:1725 read 0x7ed9595 thru 0x7ee31d0 (0x9c3c bytes) 2024-09-13T09:17:59.627 INFO:teuthology.orchestra.run.smithi032.stdout:1726 write 0x686ae04 thru 0x6879e61 (0xf05e bytes) 2024-09-13T09:17:59.638 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:17:59.634+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:17:59.640 INFO:teuthology.orchestra.run.smithi032.stdout:1727 writesame 0x520b080 thru 0x520ee4f data_size 0x56(0x3dd0 bytes) 2024-09-13T09:17:59.647 INFO:teuthology.orchestra.run.smithi032.stdout:1728 write 0x696eb64 thru 0x6976b93 (0x8030 bytes) 2024-09-13T09:17:59.661 INFO:teuthology.orchestra.run.smithi032.stdout:1729 writesame 0xe84bfd5 thru 0xe85784b data_size 0x51(0xb877 bytes) 2024-09-13T09:17:59.661 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:59.661 INFO:teuthology.orchestra.run.smithi032.stdout:1731 writesame 0xda8e91d thru 0xda8f61e data_size 0x1e(0xd02 bytes) 2024-09-13T09:17:59.665 INFO:teuthology.orchestra.run.smithi032.stdout:1733 writesame 0x2779eef thru 0x27806ec data_size 0x22(0x67fe bytes) 2024-09-13T09:17:59.674 INFO:teuthology.orchestra.run.smithi032.stdout:1734 write 0x26c7c2e thru 0x26c8f79 (0x134c bytes) 2024-09-13T09:17:59.679 INFO:teuthology.orchestra.run.smithi032.stdout:1737 read 0x3fd1bd9 thru 0x3fd3297 (0x16bf bytes) 2024-09-13T09:17:59.679 INFO:teuthology.orchestra.run.smithi032.stdout:1739 write 0x50f87bf thru 0x5105cb5 (0xd4f7 bytes) 2024-09-13T09:17:59.689 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:59.689 INFO:teuthology.orchestra.run.smithi032.stdout:1742 write 0xe3dd6ce thru 0xe3e9aff (0xc432 bytes) 2024-09-13T09:17:59.701 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:59.702 INFO:teuthology.orchestra.run.smithi032.stdout:1746 write 0x20a99e7 thru 0x20b1882 (0x7e9c bytes) 2024-09-13T09:17:59.722 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:17:59.718+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:17:59.733 INFO:teuthology.orchestra.run.smithi032.stdout:1747 trunc from 0xe85784c to 0x6687739 2024-09-13T09:17:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:59.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:17:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:1751 write 0x7fbef3a thru 0x7fca794 (0xb85b bytes) 2024-09-13T09:17:59.811 INFO:teuthology.orchestra.run.smithi032.stdout:1752 write 0x9b5ab30 thru 0x9b5bcb6 (0x1187 bytes) 2024-09-13T09:17:59.819 INFO:teuthology.orchestra.run.smithi032.stdout:1753 trunc from 0x9b5bcb7 to 0xaad7b21 2024-09-13T09:17:59.822 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:59.820+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:59.829 INFO:teuthology.orchestra.run.smithi032.stdout:1754 write 0x57097a3 thru 0x57156e7 (0xbf45 bytes) 2024-09-13T09:17:59.842 INFO:teuthology.orchestra.run.smithi032.stdout:1755 read 0x2797a0c thru 0x2799880 (0x1e75 bytes) 2024-09-13T09:17:59.843 INFO:teuthology.orchestra.run.smithi032.stdout:1756 read 0x54d1f8d thru 0x54dcdb1 (0xae25 bytes) 2024-09-13T09:17:59.852 INFO:teuthology.orchestra.run.smithi032.stdout:1758 trunc from 0xaad7b21 to 0x789ef6e 2024-09-13T09:17:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:17:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:1762 read 0x1da1a3c thru 0x1da6dfe (0x53c3 bytes) 2024-09-13T09:17:59.884 INFO:teuthology.orchestra.run.smithi032.stdout:1763 trunc from 0x789ef6e to 0x2950e36 2024-09-13T09:17:59.923 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:17:59.920+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:17:59.926 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:17:59.932 INFO:teuthology.orchestra.run.smithi032.stdout:1764 write 0xec0ef8 thru 0xecaf00 (0xa009 bytes) 2024-09-13T09:17:59.948 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:17:59.948 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:17:59.963 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:00.025 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:00.018+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:00.085 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:00.085 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:00.100 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:00.125 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:00.124+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:00.226 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:00.224+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:00.262 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:00.262 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:00.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:00.328 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:00.322+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:00.405 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:00.405 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:00.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:00.429 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:00.424+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:00.530 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:00.528+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:00.592 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:00.592 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:00.606 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:00.654 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:00.650+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:00.698 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:00.731 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:00.726+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:00.732 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:00.732 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:00.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:00.833 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:00.826+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:00.859 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:00.860 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:00.876 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:00.876 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:00.879 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:00.879 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:00.879 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:00.879 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:00.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:00.894 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:00.894 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:00.910 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:00.911 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:00.933 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:00.928+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:00.936 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:00.936 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:00.957 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:00.958 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:00.962 INFO:teuthology.orchestra.run.smithi032.stdout:1765 punch from 0x25af523 to 0x25af69c, (0x179 bytes) 2024-09-13T09:18:00.963 INFO:teuthology.orchestra.run.smithi032.stdout:1766 read 0x1c5f533 thru 0x1c6814c (0x8c1a bytes) 2024-09-13T09:18:00.966 INFO:teuthology.orchestra.run.smithi032.stdout:1768 trunc from 0x2950e36 to 0xe1bf7 2024-09-13T09:18:00.983 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:01.009 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:01.010 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:01.028 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:01.028 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:01.028 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:01.028 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:01.029 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:01.035 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:01.032+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:01.037 INFO:teuthology.orchestra.run.smithi032.stdout:1769 read 0x95fc8 thru 0x9f95a (0x9993 bytes) 2024-09-13T09:18:01.038 INFO:teuthology.orchestra.run.smithi032.stdout:1770 punch from 0x8d8bb to 0x97f09, (0xa64e bytes) 2024-09-13T09:18:01.038 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:01.038 INFO:teuthology.orchestra.run.smithi032.stdout:1774 punch from 0x6e288 to 0x7897a, (0xa6f2 bytes) 2024-09-13T09:18:01.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:01.069 INFO:teuthology.orchestra.run.smithi032.stdout:1776 write 0xb1a0aaf thru 0xb1af5e9 (0xeb3b bytes) 2024-09-13T09:18:01.094 INFO:teuthology.orchestra.run.smithi032.stdout:1779 writesame 0x908b56f thru 0x909323e data_size 0x8(0x7cd0 bytes) 2024-09-13T09:18:01.124 INFO:teuthology.orchestra.run.smithi032.stdout:1781 writesame 0xb95130b thru 0xb95dbc6 data_size 0x3a(0xc8bc bytes) 2024-09-13T09:18:01.124 INFO:teuthology.orchestra.run.smithi032.stdout:1782 writesame 0x8658754 thru 0x8659fcb data_size 0x3a(0x1878 bytes) 2024-09-13T09:18:01.130 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:01.130 INFO:teuthology.orchestra.run.smithi032.stdout:1784 read 0xd74cd9 thru 0xd753dc (0x704 bytes) 2024-09-13T09:18:01.130 INFO:teuthology.orchestra.run.smithi032.stdout:1785 write 0x26d2163 thru 0x26dc895 (0xa733 bytes) 2024-09-13T09:18:01.135 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:01.132+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:01.148 INFO:teuthology.orchestra.run.smithi032.stdout:1786 trunc from 0xb95dbc7 to 0xaa0385c 2024-09-13T09:18:01.170 INFO:teuthology.orchestra.run.smithi032.stdout:1787 read 0x74346b4 thru 0x743c552 (0x7e9f bytes) 2024-09-13T09:18:01.171 INFO:teuthology.orchestra.run.smithi032.stdout:1788 write 0x21bd45c thru 0x21c9540 (0xc0e5 bytes) 2024-09-13T09:18:01.198 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:01.199 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:01.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:01.199 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:01.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.199 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3:ed699f0f:::rbd_data.118591b29fad.000000000000008c:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:18:01.199 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:18:01.199 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.336162495, 2024-09-13T09:18:01.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94878089700000001, 2024-09-13T09:18:01.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.200 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:18:01.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.200 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:18:01.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:18:01.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:18:01.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:18:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:18:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:18:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:18:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:18:01.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:18:01.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:18:01.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:18:01.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:18:01.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:18:01.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:18:01.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810654+0000", 2024-09-13T09:18:01.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94834837599999999 2024-09-13T09:18:01.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810755+0000", 2024-09-13T09:18:01.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100987 2024-09-13T09:18:01.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810828+0000", 2024-09-13T09:18:01.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2751999999999997e-05 2024-09-13T09:18:01.205 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.205 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.205 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.206 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4104 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.206 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:18:01.206 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224684737, 2024-09-13T09:18:01.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023862547000000001, 2024-09-13T09:18:01.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.206 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.206 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.207 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.207 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4104 2024-09-13T09:18:01.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.208 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:18:01.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:18:01.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973524+0000", 2024-09-13T09:18:01.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973527+0000", 2024-09-13T09:18:01.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1200000000000002e-06 2024-09-13T09:18:01.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973535+0000", 2024-09-13T09:18:01.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2479999999999999e-06 2024-09-13T09:18:01.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973548+0000", 2024-09-13T09:18:01.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.365e-05 2024-09-13T09:18:01.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997205+0000", 2024-09-13T09:18:01.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023657088999999999 2024-09-13T09:18:01.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997289+0000", 2024-09-13T09:18:01.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3801000000000006e-05 2024-09-13T09:18:01.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997387+0000", 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8317000000000001e-05 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4105 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.22465622, 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024049968000000001, 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.213 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.214 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.214 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.214 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4105 2024-09-13T09:18:01.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.214 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:18:01.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:18:01.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:18:01.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973556+0000", 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7420000000000002e-06 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973564+0000", 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973569+0000", 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-13T09:18:01.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997418+0000", 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023849052999999999 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997504+0000", 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6075999999999994e-05 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997603+0000", 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9455000000000002e-05 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4106 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224636628999999, 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024364336, 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.218 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4106 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:18:01.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:18:01.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973575+0000", 2024-09-13T09:18:01.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.322e-06 2024-09-13T09:18:01.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973594+0000", 2024-09-13T09:18:01.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9031000000000001e-05 2024-09-13T09:18:01.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973601+0000", 2024-09-13T09:18:01.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5300000000000002e-06 2024-09-13T09:18:01.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997648+0000", 2024-09-13T09:18:01.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024047731999999999 2024-09-13T09:18:01.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997748+0000", 2024-09-13T09:18:01.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9807000000000002e-05 2024-09-13T09:18:01.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997937+0000", 2024-09-13T09:18:01.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018924299999999999 2024-09-13T09:18:01.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.223 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.223 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4107 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.223 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:18:01.224 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224604185, 2024-09-13T09:18:01.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542690999999998, 2024-09-13T09:18:01.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.225 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.225 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4107 2024-09-13T09:18:01.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.225 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:18:01.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973614+0000", 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973621+0000", 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0110000000000003e-06 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973626+0000", 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0320000000000003e-06 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997979+0000", 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024353063000000001 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998059+0000", 2024-09-13T09:18:01.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9221000000000006e-05 2024-09-13T09:18:01.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998148+0000", 2024-09-13T09:18:01.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9506999999999995e-05 2024-09-13T09:18:01.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4108 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224578576999999, 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024717223999999999, 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4108 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973633+0000", 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2089999999999999e-06 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973639+0000", 2024-09-13T09:18:01.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1469999999999998e-06 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973644+0000", 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5310000000000003e-06 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998175+0000", 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024531826999999999 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998257+0000", 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1668999999999993e-05 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998348+0000", 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1184e-05 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4109 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.233 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:18:01.234 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224559046, 2024-09-13T09:18:01.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024935052999999999, 2024-09-13T09:18:01.234 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.234 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.235 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4109 2024-09-13T09:18:01.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.235 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:18:01.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:18:01.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973650+0000", 2024-09-13T09:18:01.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973653+0000", 2024-09-13T09:18:01.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.328e-06 2024-09-13T09:18:01.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973659+0000", 2024-09-13T09:18:01.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3269999999999997e-06 2024-09-13T09:18:01.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973664+0000", 2024-09-13T09:18:01.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8149999999999996e-06 2024-09-13T09:18:01.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998375+0000", 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024711767999999999 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998484+0000", 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108257 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998586+0000", 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101953 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4110 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224541711000001, 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025120632, 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.240 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.240 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4110 2024-09-13T09:18:01.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.240 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:18:01.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:18:01.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:18:01.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973678+0000", 2024-09-13T09:18:01.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0712e-05 2024-09-13T09:18:01.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973685+0000", 2024-09-13T09:18:01.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9029999999999998e-06 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973690+0000", 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.955e-06 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998613+0000", 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024922546 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998695+0000", 2024-09-13T09:18:01.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2066999999999999e-05 2024-09-13T09:18:01.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998789+0000", 2024-09-13T09:18:01.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3813000000000005e-05 2024-09-13T09:18:01.244 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.244 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.244 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.245 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4111 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.245 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:18:01.245 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224514873, 2024-09-13T09:18:01.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025376396999999998, 2024-09-13T09:18:01.245 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.245 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.245 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.245 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.245 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.245 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4111 2024-09-13T09:18:01.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.246 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:18:01.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:18:01.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973694+0000", 2024-09-13T09:18:01.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973697+0000", 2024-09-13T09:18:01.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.384e-06 2024-09-13T09:18:01.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973703+0000", 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9970000000000004e-06 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973707+0000", 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7169999999999999e-06 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998817+0000", 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025109446000000001 2024-09-13T09:18:01.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998898+0000", 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1003999999999999e-05 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999071+0000", 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000173337 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.250 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4112 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.250 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:18:01.250 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224498049000001, 2024-09-13T09:18:01.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025596280999999999, 2024-09-13T09:18:01.250 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.250 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.250 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.250 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.250 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.251 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4112 2024-09-13T09:18:01.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.251 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:18:01.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:18:01.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973711+0000", 2024-09-13T09:18:01.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973713+0000", 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2670000000000001e-06 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973719+0000", 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9379999999999997e-06 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973724+0000", 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6809999999999997e-06 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999115+0000", 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025390511000000001 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999202+0000", 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7787999999999994e-05 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999308+0000", 2024-09-13T09:18:01.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105568 2024-09-13T09:18:01.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.255 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.255 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4113 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 430080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.255 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:18:01.255 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224480904, 2024-09-13T09:18:01.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025814090000000001, 2024-09-13T09:18:01.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.255 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.255 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.256 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4113 2024-09-13T09:18:01.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.256 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:18:01.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973728+0000", 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973738+0000", 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4339999999999998e-06 2024-09-13T09:18:01.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973745+0000", 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9639999999999997e-06 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973754+0000", 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4760000000000004e-06 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999360+0000", 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025606035999999999 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999443+0000", 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3393000000000006e-05 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999543+0000", 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9351000000000005e-05 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.259 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4114 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224451048999999, 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026036354000000001, 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4114 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.260 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:18:01.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:18:01.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973758+0000", 2024-09-13T09:18:01.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973761+0000", 2024-09-13T09:18:01.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-13T09:18:01.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973767+0000", 2024-09-13T09:18:01.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6209999999999998e-06 2024-09-13T09:18:01.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973772+0000", 2024-09-13T09:18:01.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:18:01.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999593+0000", 2024-09-13T09:18:01.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025820532 2024-09-13T09:18:01.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999691+0000", 2024-09-13T09:18:01.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8701000000000002e-05 2024-09-13T09:18:01.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999795+0000", 2024-09-13T09:18:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103467 2024-09-13T09:18:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4115 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:18:01.266 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224433306000002, 2024-09-13T09:18:01.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026260549000000001, 2024-09-13T09:18:01.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.266 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.266 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.266 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.266 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.266 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4115 2024-09-13T09:18:01.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.267 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:18:01.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:18:01.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:18:01.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973778+0000", 2024-09-13T09:18:01.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.091e-06 2024-09-13T09:18:01.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973784+0000", 2024-09-13T09:18:01.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.268e-06 2024-09-13T09:18:01.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973789+0000", 2024-09-13T09:18:01.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4599999999999996e-06 2024-09-13T09:18:01.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999836+0000", 2024-09-13T09:18:01.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026046980000000001 2024-09-13T09:18:01.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999932+0000", 2024-09-13T09:18:01.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5863e-05 2024-09-13T09:18:01.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000037+0000", 2024-09-13T09:18:01.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105264 2024-09-13T09:18:01.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.272 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.273 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4116 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.273 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:18:01.273 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224416917999999, 2024-09-13T09:18:01.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026488986999999999, 2024-09-13T09:18:01.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.273 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.273 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4116 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:18:01.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973792+0000", 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973794+0000", 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8950000000000001e-06 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973800+0000", 2024-09-13T09:18:01.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9780000000000002e-06 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973811+0000", 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1192999999999999e-05 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000081+0000", 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026269942000000001 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000175+0000", 2024-09-13T09:18:01.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3819000000000004e-05 2024-09-13T09:18:01.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000282+0000", 2024-09-13T09:18:01.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106507 2024-09-13T09:18:01.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.277 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4117 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224393374000002, 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026706133999999999, 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4117 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973818+0000", 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6259999999999999e-06 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973825+0000", 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.737e-06 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973830+0000", 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4399999999999998e-06 2024-09-13T09:18:01.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000317+0000", 2024-09-13T09:18:01.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026486948 2024-09-13T09:18:01.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000398+0000", 2024-09-13T09:18:01.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1160000000000002e-05 2024-09-13T09:18:01.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000522+0000", 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012466800000000001 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4118 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224375845000001, 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026891327, 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.282 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.283 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4118 2024-09-13T09:18:01.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.283 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:18:01.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973833+0000", 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973836+0000", 2024-09-13T09:18:01.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2350000000000002e-06 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973841+0000", 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7230000000000001e-06 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973846+0000", 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1829999999999996e-06 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000563+0000", 2024-09-13T09:18:01.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026717797000000001 2024-09-13T09:18:01.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000630+0000", 2024-09-13T09:18:01.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6152e-05 2024-09-13T09:18:01.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000725+0000", 2024-09-13T09:18:01.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5601999999999997e-05 2024-09-13T09:18:01.287 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.287 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.287 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.287 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4119 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.287 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:18:01.287 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.224359907, 2024-09-13T09:18:01.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027064775999999999, 2024-09-13T09:18:01.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4119 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:18:01.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973849+0000", 2024-09-13T09:18:01.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973852+0000", 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973859+0000", 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7589999999999997e-06 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973864+0000", 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8119999999999998e-06 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000747+0000", 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026882653999999999 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000816+0000", 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8841e-05 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000915+0000", 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8933999999999998e-05 2024-09-13T09:18:01.291 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.292 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.292 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.292 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14583 3.1f 3:feba6951:::rbd_data.11853ecd5f5f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:01.292 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:18:01.292 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.664179231, 2024-09-13T09:18:01.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.050883949999999997, 2024-09-13T09:18:01.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.292 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:01.292 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.292 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.293 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.293 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14583 2024-09-13T09:18:01.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.293 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:18:01.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:18:01.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534028+0000", 2024-09-13T09:18:01.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:01.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538304+0000", 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00427634 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538342+0000", 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7567000000000001e-05 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538357+0000", 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4936999999999999e-05 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538411+0000", 2024-09-13T09:18:01.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4166000000000003e-05 2024-09-13T09:18:01.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538539+0000", 2024-09-13T09:18:01.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000127966 2024-09-13T09:18:01.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:01.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.539016+0000", 2024-09-13T09:18:01.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477014 2024-09-13T09:18:01.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:01.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.547890+0000", 2024-09-13T09:18:01.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0088736430000000005 2024-09-13T09:18:01.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:01.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584825+0000", 2024-09-13T09:18:01.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036934966 2024-09-13T09:18:01.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584866+0000", 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1514000000000003e-05 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584914+0000", 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8167000000000001e-05 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.298 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16399 3.1f 3:fcbf421e:::rbd_data.1185f20bdb42.0000000000000019:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.655128+0000", 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.5430815770000006, 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042135120999999998, 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16399 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655128+0000", 2024-09-13T09:18:01.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655128+0000", 2024-09-13T09:18:01.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655126+0000", 2024-09-13T09:18:01.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:01.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655139+0000", 2024-09-13T09:18:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3373000000000001e-05 2024-09-13T09:18:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655164+0000", 2024-09-13T09:18:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4576999999999999e-05 2024-09-13T09:18:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655177+0000", 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2853e-05 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655257+0000", 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0445999999999995e-05 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655382+0000", 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012448900000000001 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.663115+0000", 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077335030000000001 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.697103+0000", 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033987512999999997 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.697165+0000", 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2731000000000006e-05 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:01.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.697223+0000", 2024-09-13T09:18:01.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7651000000000001e-05 2024-09-13T09:18:01.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.697263+0000", 2024-09-13T09:18:01.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0061999999999997e-05 2024-09-13T09:18:01.304 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.304 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.304 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.305 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19465 3.10 3:0e3a5b30:::rbd_data.1185b72576e4.0000000000000062:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:01.305 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:18:01.305 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2503177579999996, 2024-09-13T09:18:01.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97489159000000003, 2024-09-13T09:18:01.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:01.305 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.305 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19465 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:18:01.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947890+0000", 2024-09-13T09:18:01.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:01.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947899+0000", 2024-09-13T09:18:01.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6840000000000008e-06 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947917+0000", 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7731e-05 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947925+0000", 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4189999999999996e-06 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947990+0000", 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5909999999999997e-05 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947998+0000", 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4449999999999999e-06 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.919994+0000", 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97199595900000002 2024-09-13T09:18:01.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920258+0000", 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026439500000000001 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920629+0000", 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037096299999999998 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.921797+0000", 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001167721 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922544+0000", 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00074661200000000001 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922628+0000", 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4016000000000003e-05 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922783+0000", 2024-09-13T09:18:01.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015585499999999999 2024-09-13T09:18:01.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.312 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.312 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.312 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:01.313 INFO:teuthology.orchestra.run.smithi032.stdout:1789 trunc from 0xaa0385c to 0x729f8fb 2024-09-13T09:18:01.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:01.315 INFO:teuthology.orchestra.run.smithi032.stdout:1790 read 0x31cbe0 thru 0x324e65 (0x8286 bytes) 2024-09-13T09:18:01.315 INFO:teuthology.orchestra.run.smithi032.stdout:1791 read 0x6664006 thru 0x666ceaa (0x8ea5 bytes) 2024-09-13T09:18:01.315 INFO:teuthology.orchestra.run.smithi032.stdout:1792 write 0xdcc3c72 thru 0xdcc9bda (0x5f69 bytes) 2024-09-13T09:18:01.316 INFO:teuthology.orchestra.run.smithi032.stdout:1794 writesame 0x7cd82ef thru 0x7cdd87d data_size 0x7(0x558f bytes) 2024-09-13T09:18:01.316 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:01.316 INFO:teuthology.orchestra.run.smithi032.stdout:1797 writesame 0xcbe2a4 thru 0xccce1a data_size 0x3(0xeb77 bytes) 2024-09-13T09:18:01.316 INFO:teuthology.orchestra.run.smithi032.stdout:1798 read 0x9c42a1b thru 0x9c44d20 (0x2306 bytes) 2024-09-13T09:18:01.316 INFO:teuthology.orchestra.run.smithi032.stdout:1799 write 0x69e5a6b thru 0x69efa92 (0xa028 bytes) 2024-09-13T09:18:01.316 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:01.310+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:01.326 INFO:teuthology.orchestra.run.smithi032.stdout:1802 write 0x1ea6f36 thru 0x1eb161b (0xa6e6 bytes) 2024-09-13T09:18:01.350 INFO:teuthology.orchestra.run.smithi032.stdout:1805 trunc from 0xdcc9bdb to 0xaee35e7 2024-09-13T09:18:01.382 INFO:teuthology.orchestra.run.smithi032.stdout:1806 trunc from 0xaee35e7 to 0x696859e 2024-09-13T09:18:01.414 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:01.410+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:01.430 INFO:teuthology.orchestra.run.smithi032.stdout:1811 writesame 0xacd46b2 thru 0xacda6cf data_size 0x9(0x601e bytes) 2024-09-13T09:18:01.443 INFO:teuthology.orchestra.run.smithi032.stdout:1812 write 0xb0e9766 thru 0xb0f46a4 (0xaf3f bytes) 2024-09-13T09:18:01.459 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:01.459 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:01.459 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:01.459 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:01.462 INFO:teuthology.orchestra.run.smithi032.stdout:1814 read 0x214bef7 thru 0x215b8c0 (0xf9ca bytes) 2024-09-13T09:18:01.477 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:01.480 INFO:teuthology.orchestra.run.smithi032.stdout:1815 write 0xcb8187c thru 0xcb8d66d (0xbdf2 bytes) 2024-09-13T09:18:01.503 INFO:teuthology.orchestra.run.smithi032.stdout:1816 punch from 0x979fd79 to 0x97a5b42, (0x5dc9 bytes) 2024-09-13T09:18:01.503 INFO:teuthology.orchestra.run.smithi032.stdout:1817 read 0xaa68108 thru 0xaa761b1 (0xe0aa bytes) 2024-09-13T09:18:01.503 INFO:teuthology.orchestra.run.smithi032.stdout:1818 write 0x6e7d5c9 thru 0x6e865f7 (0x902f bytes) 2024-09-13T09:18:01.515 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:01.510+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:01.518 INFO:teuthology.orchestra.run.smithi032.stdout:1819 read 0x57c893a thru 0x57d2281 (0x9948 bytes) 2024-09-13T09:18:01.518 INFO:teuthology.orchestra.run.smithi032.stdout:1820 read 0xc399104 thru 0xc39caa3 (0x39a0 bytes) 2024-09-13T09:18:01.518 INFO:teuthology.orchestra.run.smithi032.stdout:1822 read 0x166d2a3 thru 0x167cad2 (0xf830 bytes) 2024-09-13T09:18:01.518 INFO:teuthology.orchestra.run.smithi032.stdout:1823 punch from 0xb071886 to 0xb0795b1, (0x7d2b bytes) 2024-09-13T09:18:01.518 INFO:teuthology.orchestra.run.smithi032.stdout:1824 write 0x6bc8ecf thru 0x6bcc076 (0x31a8 bytes) 2024-09-13T09:18:01.521 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:01.521 INFO:teuthology.orchestra.run.smithi032.stdout:1828 punch from 0xb902d67 to 0xb90c2f9, (0x9592 bytes) 2024-09-13T09:18:01.521 INFO:teuthology.orchestra.run.smithi032.stdout:1829 punch from 0xa4307c6 to 0xa432d61, (0x259b bytes) 2024-09-13T09:18:01.521 INFO:teuthology.orchestra.run.smithi032.stdout:1830 write 0x37b82d0 thru 0x37bf21d (0x6f4e bytes) 2024-09-13T09:18:01.531 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:01.531 INFO:teuthology.orchestra.run.smithi032.stdout:1832 trunc from 0xcb8d66e to 0xcb44da5 2024-09-13T09:18:01.545 INFO:teuthology.orchestra.run.smithi032.stdout:1833 read 0x871e4a9 thru 0x872c36d (0xdec5 bytes) 2024-09-13T09:18:01.545 INFO:teuthology.orchestra.run.smithi032.stdout:1834 read 0x8dc52d8 thru 0x8dc7f2f (0x2c58 bytes) 2024-09-13T09:18:01.546 INFO:teuthology.orchestra.run.smithi032.stdout:1836 read 0x3f53fe6 thru 0x3f5e30a (0xa325 bytes) 2024-09-13T09:18:01.546 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:01.556 INFO:teuthology.orchestra.run.smithi032.stdout:1838 write 0xdcaac5e thru 0xdcb50ba (0xa45d bytes) 2024-09-13T09:18:01.570 INFO:teuthology.orchestra.run.smithi032.stdout:1839 writesame 0x9731f6b thru 0x973f8c6 data_size 0x6(0xd95c bytes) 2024-09-13T09:18:01.585 INFO:teuthology.orchestra.run.smithi032.stdout:1842 read 0xb7676e2 thru 0xb76950f (0x1e2e bytes) 2024-09-13T09:18:01.585 INFO:teuthology.orchestra.run.smithi032.stdout:1843 punch from 0xc546882 to 0xc55621c, (0xf99a bytes) 2024-09-13T09:18:01.585 INFO:teuthology.orchestra.run.smithi032.stdout:1845 write 0x38e0f84 thru 0x38e5b3e (0x4bbb bytes) 2024-09-13T09:18:01.601 INFO:teuthology.orchestra.run.smithi032.stdout:1847 trunc from 0xdcb50bb to 0x84bfa72 2024-09-13T09:18:01.615 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:01.612+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:01.623 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:01.623 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:01.623 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:01.623 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:01.623 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:01.641 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:01.650 INFO:teuthology.orchestra.run.smithi032.stdout:1848 write 0xd8a8ab2 thru 0xd8b1063 (0x85b2 bytes) 2024-09-13T09:18:01.663 INFO:teuthology.orchestra.run.smithi032.stdout:1850 write 0x8dffa75 thru 0x8dfffe9 (0x575 bytes) 2024-09-13T09:18:01.668 INFO:teuthology.orchestra.run.smithi032.stdout:1851 punch from 0x16881b0 to 0x16944cc, (0xc31c bytes) 2024-09-13T09:18:01.668 INFO:teuthology.orchestra.run.smithi032.stdout:1852 writesame 0x779f2a thru 0x77a9a0 data_size 0x13(0xa77 bytes) 2024-09-13T09:18:01.674 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:01.674 INFO:teuthology.orchestra.run.smithi032.stdout:1856 clone 12 order 23 su 524288 sc 7 2024-09-13T09:18:01.717 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:01.710+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:01.777 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:01.778 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:01.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:01.778 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:01.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.778 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5478 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.778 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:18:01.778 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.237258534999999, 2024-09-13T09:18:01.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027749532, 2024-09-13T09:18:01.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5478 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539236+0000", 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.041e-06 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539242+0000", 2024-09-13T09:18:01.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0650000000000004e-06 2024-09-13T09:18:01.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539246+0000", 2024-09-13T09:18:01.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1500000000000001e-06 2024-09-13T09:18:01.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566770+0000", 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027523178999999998 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566846+0000", 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5941999999999999e-05 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566984+0000", 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000138486 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5479 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.237242271, 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027937460000000001, 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5479 2024-09-13T09:18:01.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.784 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:18:01.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:18:01.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539250+0000", 2024-09-13T09:18:01.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539256+0000", 2024-09-13T09:18:01.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:18:01.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539263+0000", 2024-09-13T09:18:01.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3779999999999998e-06 2024-09-13T09:18:01.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539273+0000", 2024-09-13T09:18:01.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0457e-05 2024-09-13T09:18:01.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567010+0000", 2024-09-13T09:18:01.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027736878 2024-09-13T09:18:01.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567083+0000", 2024-09-13T09:18:01.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3187000000000002e-05 2024-09-13T09:18:01.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567188+0000", 2024-09-13T09:18:01.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010491499999999999 2024-09-13T09:18:01.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.791 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5480 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.791 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.237215053, 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028132641999999999, 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5480 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:01.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:01.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539280+0000", 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539297+0000", 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7147999999999999e-05 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539303+0000", 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9039999999999997e-06 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567230+0000", 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027927250000000001 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567316+0000", 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5970999999999994e-05 2024-09-13T09:18:01.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567411+0000", 2024-09-13T09:18:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4523000000000003e-05 2024-09-13T09:18:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5481 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:18:01.796 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.237185057000001, 2024-09-13T09:18:01.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028317978000000001, 2024-09-13T09:18:01.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.797 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.797 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5481 2024-09-13T09:18:01.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.798 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:18:01.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:18:01.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539307+0000", 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539311+0000", 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.264e-06 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539317+0000", 2024-09-13T09:18:01.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539322+0000", 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6949999999999999e-06 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567446+0000", 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028124341000000001 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567527+0000", 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0785000000000007e-05 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567626+0000", 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9103000000000003e-05 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.801 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5482 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:18:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.237167232000001, 2024-09-13T09:18:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028619326, 2024-09-13T09:18:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5482 2024-09-13T09:18:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539325+0000", 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539333+0000", 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.055e-06 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539340+0000", 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4980000000000003e-06 2024-09-13T09:18:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539344+0000", 2024-09-13T09:18:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5369999999999999e-06 2024-09-13T09:18:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567655+0000", 2024-09-13T09:18:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028310906 2024-09-13T09:18:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567732+0000", 2024-09-13T09:18:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6829999999999995e-05 2024-09-13T09:18:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567945+0000", 2024-09-13T09:18:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000212909 2024-09-13T09:18:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5483 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.237143970999998, 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028784015999999999, 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5483 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:01.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539351+0000", 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539357+0000", 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0090000000000004e-06 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539361+0000", 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.357e-06 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567974+0000", 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028612385000000001 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568050+0000", 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5820000000000003e-05 2024-09-13T09:18:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568133+0000", 2024-09-13T09:18:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3551e-05 2024-09-13T09:18:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5484 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:18:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.237127419, 2024-09-13T09:18:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028949923999999998, 2024-09-13T09:18:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5484 2024-09-13T09:18:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:18:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:18:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539365+0000", 2024-09-13T09:18:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539367+0000", 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539374+0000", 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539378+0000", 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568159+0000", 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028780490999999998 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568236+0000", 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7156000000000005e-05 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568316+0000", 2024-09-13T09:18:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9598999999999995e-05 2024-09-13T09:18:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5485 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236307539999999, 2024-09-13T09:18:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028332593, 2024-09-13T09:18:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5485 2024-09-13T09:18:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540189+0000", 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540198+0000", 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9099999999999994e-06 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540206+0000", 2024-09-13T09:18:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-13T09:18:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568360+0000", 2024-09-13T09:18:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028154324000000001 2024-09-13T09:18:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568436+0000", 2024-09-13T09:18:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5821000000000005e-05 2024-09-13T09:18:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568518+0000", 2024-09-13T09:18:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1895000000000001e-05 2024-09-13T09:18:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5486 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236254531, 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028479536999999999, 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5486 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540241+0000", 2024-09-13T09:18:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4290000000000001e-06 2024-09-13T09:18:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540249+0000", 2024-09-13T09:18:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9480000000000008e-06 2024-09-13T09:18:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540255+0000", 2024-09-13T09:18:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6019999999999999e-06 2024-09-13T09:18:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568543+0000", 2024-09-13T09:18:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028288260999999999 2024-09-13T09:18:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568636+0000", 2024-09-13T09:18:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2627999999999995e-05 2024-09-13T09:18:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568718+0000", 2024-09-13T09:18:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2213999999999996e-05 2024-09-13T09:18:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.828 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5487 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.828 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:18:01.828 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236233446, 2024-09-13T09:18:01.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028653435000000001, 2024-09-13T09:18:01.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.829 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.829 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.829 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.829 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5487 2024-09-13T09:18:01.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.829 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:18:01.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:18:01.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540259+0000", 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540262+0000", 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.413e-06 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540268+0000", 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3940000000000001e-06 2024-09-13T09:18:01.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540272+0000", 2024-09-13T09:18:01.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3989999999999997e-06 2024-09-13T09:18:01.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568745+0000", 2024-09-13T09:18:01.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028472173999999999 2024-09-13T09:18:01.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568822+0000", 2024-09-13T09:18:01.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7694000000000005e-05 2024-09-13T09:18:01.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568913+0000", 2024-09-13T09:18:01.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0779000000000006e-05 2024-09-13T09:18:01.833 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.833 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.833 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.834 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5488 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 430080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.834 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:01.834 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236216045999999, 2024-09-13T09:18:01.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028788326999999999, 2024-09-13T09:18:01.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.835 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5488 2024-09-13T09:18:01.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.835 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:01.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:01.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:01.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540279+0000", 2024-09-13T09:18:01.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2409999999999998e-06 2024-09-13T09:18:01.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540285+0000", 2024-09-13T09:18:01.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3180000000000003e-06 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540300+0000", 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4625e-05 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568939+0000", 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028639688 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569016+0000", 2024-09-13T09:18:01.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6556000000000004e-05 2024-09-13T09:18:01.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569065+0000", 2024-09-13T09:18:01.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9246000000000003e-05 2024-09-13T09:18:01.839 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.839 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.840 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5489 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.840 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:18:01.840 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236188452, 2024-09-13T09:18:01.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028875332, 2024-09-13T09:18:01.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5489 2024-09-13T09:18:01.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.841 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:18:01.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:18:01.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540304+0000", 2024-09-13T09:18:01.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540307+0000", 2024-09-13T09:18:01.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4370000000000001e-06 2024-09-13T09:18:01.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540313+0000", 2024-09-13T09:18:01.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:18:01.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540318+0000", 2024-09-13T09:18:01.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:18:01.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569079+0000", 2024-09-13T09:18:01.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028760714 2024-09-13T09:18:01.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569128+0000", 2024-09-13T09:18:01.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8702999999999998e-05 2024-09-13T09:18:01.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569180+0000", 2024-09-13T09:18:01.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2105999999999999e-05 2024-09-13T09:18:01.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.845 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5490 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.845 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:01.845 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236169957000001, 2024-09-13T09:18:01.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029041027000000001, 2024-09-13T09:18:01.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.845 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5490 2024-09-13T09:18:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540326+0000", 2024-09-13T09:18:01.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9840000000000001e-06 2024-09-13T09:18:01.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540332+0000", 2024-09-13T09:18:01.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9730000000000002e-06 2024-09-13T09:18:01.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540338+0000", 2024-09-13T09:18:01.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3550000000000001e-06 2024-09-13T09:18:01.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569202+0000", 2024-09-13T09:18:01.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028863521 2024-09-13T09:18:01.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569276+0000", 2024-09-13T09:18:01.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:18:01.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569364+0000", 2024-09-13T09:18:01.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7594000000000002e-05 2024-09-13T09:18:01.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.851 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.852 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5491 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.852 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:01.852 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236150573, 2024-09-13T09:18:01.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029221864, 2024-09-13T09:18:01.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.853 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.853 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5491 2024-09-13T09:18:01.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.853 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:01.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:01.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:01.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540350+0000", 2024-09-13T09:18:01.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4250000000000001e-06 2024-09-13T09:18:01.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540356+0000", 2024-09-13T09:18:01.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2960000000000004e-06 2024-09-13T09:18:01.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540361+0000", 2024-09-13T09:18:01.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:18:01.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569388+0000", 2024-09-13T09:18:01.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029026744 2024-09-13T09:18:01.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569474+0000", 2024-09-13T09:18:01.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6519000000000003e-05 2024-09-13T09:18:01.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569564+0000", 2024-09-13T09:18:01.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9969999999999994e-05 2024-09-13T09:18:01.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.858 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.858 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5492 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.858 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:18:01.858 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236127276000001, 2024-09-13T09:18:01.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029401203000000001, 2024-09-13T09:18:01.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.858 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.859 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.859 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.859 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.859 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5492 2024-09-13T09:18:01.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.859 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:18:01.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:18:01.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540365+0000", 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540368+0000", 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540374+0000", 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9349999999999999e-06 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540378+0000", 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.651e-06 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569591+0000", 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029212557 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569668+0000", 2024-09-13T09:18:01.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569767+0000", 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9061000000000006e-05 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5493 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236110411999999, 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029599424999999999, 2024-09-13T09:18:01.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.864 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.864 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.864 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.864 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.864 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5493 2024-09-13T09:18:01.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.864 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:18:01.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:18:01.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540382+0000", 2024-09-13T09:18:01.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540384+0000", 2024-09-13T09:18:01.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0430000000000002e-06 2024-09-13T09:18:01.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540390+0000", 2024-09-13T09:18:01.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:18:01.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540395+0000", 2024-09-13T09:18:01.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5990000000000004e-06 2024-09-13T09:18:01.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569795+0000", 2024-09-13T09:18:01.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029400582000000001 2024-09-13T09:18:01.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569880+0000", 2024-09-13T09:18:01.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5210999999999994e-05 2024-09-13T09:18:01.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569982+0000", 2024-09-13T09:18:01.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101584 2024-09-13T09:18:01.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.870 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5494 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.870 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:18:01.870 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236094327, 2024-09-13T09:18:01.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029770637999999999, 2024-09-13T09:18:01.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.870 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.871 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.871 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.871 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5494 2024-09-13T09:18:01.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.871 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:18:01.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:18:01.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540398+0000", 2024-09-13T09:18:01.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540406+0000", 2024-09-13T09:18:01.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:18:01.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540413+0000", 2024-09-13T09:18:01.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:18:01.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540418+0000", 2024-09-13T09:18:01.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8289999999999998e-06 2024-09-13T09:18:01.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570007+0000", 2024-09-13T09:18:01.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029588851999999999 2024-09-13T09:18:01.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570084+0000", 2024-09-13T09:18:01.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7253e-05 2024-09-13T09:18:01.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570169+0000", 2024-09-13T09:18:01.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5391999999999998e-05 2024-09-13T09:18:01.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.877 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.877 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5495 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.877 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:01.878 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236056010999999, 2024-09-13T09:18:01.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029938664, 2024-09-13T09:18:01.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.878 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.878 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5495 2024-09-13T09:18:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540440+0000", 2024-09-13T09:18:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0319999999999999e-06 2024-09-13T09:18:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540446+0000", 2024-09-13T09:18:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8549999999999996e-06 2024-09-13T09:18:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540451+0000", 2024-09-13T09:18:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:18:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570198+0000", 2024-09-13T09:18:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029746060000000001 2024-09-13T09:18:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570284+0000", 2024-09-13T09:18:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6044999999999994e-05 2024-09-13T09:18:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570376+0000", 2024-09-13T09:18:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2071000000000007e-05 2024-09-13T09:18:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5496 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236037056000001, 2024-09-13T09:18:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030121617999999999, 2024-09-13T09:18:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5496 2024-09-13T09:18:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540458+0000", 2024-09-13T09:18:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540464+0000", 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.156e-06 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540468+0000", 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570404+0000", 2024-09-13T09:18:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029935632 2024-09-13T09:18:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570488+0000", 2024-09-13T09:18:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3873000000000001e-05 2024-09-13T09:18:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570578+0000", 2024-09-13T09:18:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9757000000000001e-05 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5497 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.236020492000002, 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030305259000000001, 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5497 2024-09-13T09:18:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:18:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540472+0000", 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540474+0000", 2024-09-13T09:18:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0490000000000002e-06 2024-09-13T09:18:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540480+0000", 2024-09-13T09:18:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1970000000000001e-06 2024-09-13T09:18:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540491+0000", 2024-09-13T09:18:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0964e-05 2024-09-13T09:18:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570608+0000", 2024-09-13T09:18:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030116105000000001 2024-09-13T09:18:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570688+0000", 2024-09-13T09:18:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0143999999999997e-05 2024-09-13T09:18:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570778+0000", 2024-09-13T09:18:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0131999999999997e-05 2024-09-13T09:18:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:01.893 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:01.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:01.895 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:01.892+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:01.933 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0] out_osds: [1, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:18:01.933 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:18:01.933 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-13T09:18:01.933 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-13T09:18:01.953 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:18:02.034 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:02.032+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:02.084 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:02.085 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:02.085 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:02.085 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:02.096 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:02.092+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:02.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:02.136 INFO:teuthology.orchestra.run.smithi032.stdout:flattening image image_client.0-clone11 2024-09-13T09:18:02.198 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:02.194+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:02.258 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:02.258 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:02.259 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:02.259 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:02.259 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:02.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:02.305 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-09-13T09:18:02.304+0000 7f1d86e9f640 -1 mon.a@0(leader).osd e51 definitely_dead 0 2024-09-13T09:18:02.306 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:02.304+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:02.400 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:02.394+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:02.402 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-13T09:18:02.404 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3022 3.9 e31/22 3:91681905:::rbd_data.118591b29fad.0000000000000209:head v 31'171, mlcod=31'169)", 2024-09-13T09:18:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:18:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.934620141, 2024-09-13T09:18:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002956209, 2024-09-13T09:18:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:18:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:18:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468237+0000", 2024-09-13T09:18:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468300+0000", 2024-09-13T09:18:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3202999999999998e-05 2024-09-13T09:18:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468353+0000", 2024-09-13T09:18:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2735999999999998e-05 2024-09-13T09:18:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468374+0000", 2024-09-13T09:18:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0930000000000001e-05 2024-09-13T09:18:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468568+0000", 2024-09-13T09:18:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000194262 2024-09-13T09:18:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468627+0000", 2024-09-13T09:18:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8743000000000003e-05 2024-09-13T09:18:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.471132+0000", 2024-09-13T09:18:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025055540000000001 2024-09-13T09:18:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.471196+0000", 2024-09-13T09:18:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3590999999999995e-05 2024-09-13T09:18:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "pg_update_log_missing(3.4 epoch 35/22 rep_tid 113 entries 35'31 (0'0) error 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head by client.4485.0:3311 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 31'29)", 2024-09-13T09:18:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:18:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.874939628, 2024-09-13T09:18:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033204409999999999, 2024-09-13T09:18:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:18:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:18:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527918+0000", 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527943+0000", 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4752999999999998e-05 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527988+0000", 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4693000000000001e-05 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528029+0000", 2024-09-13T09:18:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1155999999999997e-05 2024-09-13T09:18:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528194+0000", 2024-09-13T09:18:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164792 2024-09-13T09:18:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.531241+0000", 2024-09-13T09:18:02.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003046867 2024-09-13T09:18:02.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.414 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.414 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3323 3.4 e35/22 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head v 35'34, mlcod=35'31)", 2024-09-13T09:18:02.414 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:18:02.414 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.867064702, 2024-09-13T09:18:02.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037621640000000001, 2024-09-13T09:18:02.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.415 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:18:02.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:18:02.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535793+0000", 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535811+0000", 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8170000000000001e-05 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535840+0000", 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9105999999999999e-05 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535877+0000", 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7017000000000002e-05 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536001+0000", 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012342899999999999 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536073+0000", 2024-09-13T09:18:02.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2123e-05 2024-09-13T09:18:02.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539433+0000", 2024-09-13T09:18:02.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033596630000000001 2024-09-13T09:18:02.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539557+0000", 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012470600000000001 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:15977 3.4 3:27c24f46:::rbd_data.11856af9a4d3.0000000000000005:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.407795189, 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003633163, 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:02.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15977 2024-09-13T09:18:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:18:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995063+0000", 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995071+0000", 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3930000000000002e-06 2024-09-13T09:18:02.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995086+0000", 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4970999999999999e-05 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995095+0000", 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8810000000000002e-06 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995180+0000", 2024-09-13T09:18:02.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5012000000000005e-05 2024-09-13T09:18:02.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995378+0000", 2024-09-13T09:18:02.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019788600000000001 2024-09-13T09:18:02.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.998698+0000", 2024-09-13T09:18:02.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003320267 2024-09-13T09:18:02.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.424 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.424 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16010 3.4 3:27c24f46:::rbd_data.11856af9a4d3.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:02.424 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:18:02.424 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.402186396999999, 2024-09-13T09:18:02.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037539579999999999, 2024-09-13T09:18:02.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.425 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16010 2024-09-13T09:18:02.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.425 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:18:02.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:18:02.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000672+0000", 2024-09-13T09:18:02.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:02.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000679+0000", 2024-09-13T09:18:02.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9120000000000001e-06 2024-09-13T09:18:02.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000693+0000", 2024-09-13T09:18:02.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3745e-05 2024-09-13T09:18:02.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000702+0000", 2024-09-13T09:18:02.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.064e-06 2024-09-13T09:18:02.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000763+0000", 2024-09-13T09:18:02.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1309000000000005e-05 2024-09-13T09:18:02.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000877+0000", 2024-09-13T09:18:02.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114214 2024-09-13T09:18:02.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:02.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.001145+0000", 2024-09-13T09:18:02.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000267765 2024-09-13T09:18:02.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:02.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.002514+0000", 2024-09-13T09:18:02.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013688730000000001 2024-09-13T09:18:02.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:02.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004301+0000", 2024-09-13T09:18:02.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017869839999999999 2024-09-13T09:18:02.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004380+0000", 2024-09-13T09:18:02.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8861000000000003e-05 2024-09-13T09:18:02.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004427+0000", 2024-09-13T09:18:02.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7713000000000002e-05 2024-09-13T09:18:02.432 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.432 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.432 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.433 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16047 3.0 3:002d813e:::rbd_data.11856af9a4d3.0000000000000014:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:02.433 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:18:02.433 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.393309394999999, 2024-09-13T09:18:02.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030667929999999999, 2024-09-13T09:18:02.433 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.433 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.434 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16047 2024-09-13T09:18:02.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.434 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:18:02.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:18:02.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009548+0000", 2024-09-13T09:18:02.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:02.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009559+0000", 2024-09-13T09:18:02.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0936e-05 2024-09-13T09:18:02.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009579+0000", 2024-09-13T09:18:02.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9976e-05 2024-09-13T09:18:02.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009590+0000", 2024-09-13T09:18:02.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1228e-05 2024-09-13T09:18:02.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009670+0000", 2024-09-13T09:18:02.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9765999999999995e-05 2024-09-13T09:18:02.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009776+0000", 2024-09-13T09:18:02.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106049 2024-09-13T09:18:02.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:18:02.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009991+0000", 2024-09-13T09:18:02.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021494999999999999 2024-09-13T09:18:02.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.011621+0000", 2024-09-13T09:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016294650000000001 2024-09-13T09:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:02.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012469+0000", 2024-09-13T09:18:02.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000848598 2024-09-13T09:18:02.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012551+0000", 2024-09-13T09:18:02.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2144999999999994e-05 2024-09-13T09:18:02.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012617+0000", 2024-09-13T09:18:02.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5827999999999994e-05 2024-09-13T09:18:02.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.443 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.444 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16226 3.4 3:22419c10:::rbd_data.1185f20bdb42.0000000000000013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:02.444 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:18:02.444 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.174309291, 2024-09-13T09:18:02.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033492510000000001, 2024-09-13T09:18:02.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.444 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.444 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.444 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.445 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16226 2024-09-13T09:18:02.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.445 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:18:02.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228546+0000", 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999957 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228569+0000", 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2347000000000001e-05 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228618+0000", 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9509999999999999e-05 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228643+0000", 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4759000000000001e-05 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228738+0000", 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5273999999999996e-05 2024-09-13T09:18:02.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228819+0000", 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0623000000000004e-05 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228921+0000", 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010215199999999999 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.230000+0000", 2024-09-13T09:18:02.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010793510000000001 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231718+0000", 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001717873 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231813+0000", 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4473000000000002e-05 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231900+0000", 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7158999999999997e-05 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16293 3.4 3:22419c10:::rbd_data.1185f20bdb42.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.02069861, 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015655326000000001, 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16293 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382158+0000", 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382176+0000", 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7863000000000002e-05 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382211+0000", 2024-09-13T09:18:02.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.481e-05 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382226+0000", 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.537e-05 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382354+0000", 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012803599999999999 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382591+0000", 2024-09-13T09:18:02.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023629099999999999 2024-09-13T09:18:02.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:02.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.385269+0000", 2024-09-13T09:18:02.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002678532 2024-09-13T09:18:02.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:02.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397515+0000", 2024-09-13T09:18:02.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012245946000000001 2024-09-13T09:18:02.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:02.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397551+0000", 2024-09-13T09:18:02.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5856999999999999e-05 2024-09-13T09:18:02.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397699+0000", 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148255 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397817+0000", 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117299 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4485.0:16293 3.4 e42/41 ondisk, result = 0)", 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:18:02.456 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.010367069999999, 2024-09-13T09:18:02.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0050343319999999999, 2024-09-13T09:18:02.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:02.457 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:18:02.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392491+0000", 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392500+0000", 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8300000000000002e-06 2024-09-13T09:18:02.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392529+0000", 2024-09-13T09:18:02.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9260000000000001e-05 2024-09-13T09:18:02.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392542+0000", 2024-09-13T09:18:02.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2969999999999999e-05 2024-09-13T09:18:02.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397468+0000", 2024-09-13T09:18:02.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0049256880000000001 2024-09-13T09:18:02.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397496+0000", 2024-09-13T09:18:02.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8209999999999999e-05 2024-09-13T09:18:02.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397527+0000", 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1276999999999999e-05 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16837 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [call rbd.sparse_copyup in=173207b,set-alloc-hint object_size 16777216 write_size 16777216,write 3145728~10909 in=10909b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.1427362569999993, 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004073887, 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.461 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16837 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260121+0000", 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260266+0000", 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014495999999999999 2024-09-13T09:18:02.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260287+0000", 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1336e-05 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260296+0000", 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6170000000000007e-06 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260352+0000", 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6252999999999998e-05 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260520+0000", 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000168381 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260807+0000", 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028624100000000001 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:02.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.262062+0000", 2024-09-13T09:18:02.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012551490000000001 2024-09-13T09:18:02.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:02.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264128+0000", 2024-09-13T09:18:02.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020660380000000001 2024-09-13T09:18:02.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264167+0000", 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9714e-05 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264198+0000", 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0105000000000001e-05 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16964 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [stat,zero 8912896~6815744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.358046+0000", 2024-09-13T09:18:02.467 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.0448137010000007, 2024-09-13T09:18:02.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027898210000000001, 2024-09-13T09:18:02.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.468 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:02.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.468 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16964 2024-09-13T09:18:02.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.468 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358046+0000", 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358046+0000", 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358044+0000", 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358055+0000", 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0958000000000001e-05 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358081+0000", 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5978000000000001e-05 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358092+0000", 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.116e-05 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358184+0000", 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2243000000000004e-05 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.358320+0000", 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013623500000000001 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.360836+0000", 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025156699999999998 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:17005 3.1a e43/41 3:5e94b56b:::rbd_data.11859d3d11dd.0000000000000008:head v 43'54, mlcod=42'52)", 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.0171715999999993, 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031754880000000002, 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.386997+0000", 2024-09-13T09:18:02.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00130933 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387013+0000", 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5752000000000001e-05 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387022+0000", 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1689999999999998e-06 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387092+0000", 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9385000000000001e-05 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387127+0000", 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5061000000000002e-05 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.388821+0000", 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001694463 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.388864+0000", 2024-09-13T09:18:02.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2721e-05 2024-09-13T09:18:02.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.476 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:17028 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [stat,zero 4194304~4718592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:18:02.476 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:18:02.476 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.9933531470000005, 2024-09-13T09:18:02.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036807770000000001, 2024-09-13T09:18:02.476 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 17028 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:18:02.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409504+0000", 2024-09-13T09:18:02.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:02.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409515+0000", 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0485e-05 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409540+0000", 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4808e-05 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409550+0000", 2024-09-13T09:18:02.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0376e-05 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409616+0000", 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6130999999999994e-05 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409725+0000", 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108614 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.413188+0000", 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003462849 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18757 3.4 3:21c7a0ac:::rbd_data.1185b72576e4.0000000000000074:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7728254549999996, 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066789190000000002, 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18757 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630032+0000", 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630044+0000", 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2245999999999999e-05 2024-09-13T09:18:02.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630067+0000", 2024-09-13T09:18:02.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3404000000000002e-05 2024-09-13T09:18:02.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630078+0000", 2024-09-13T09:18:02.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1158999999999999e-05 2024-09-13T09:18:02.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630156+0000", 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7520000000000003e-05 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630357+0000", 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020064700000000001 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.631878+0000", 2024-09-13T09:18:02.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015212940000000001 2024-09-13T09:18:02.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:02.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636574+0000", 2024-09-13T09:18:02.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046960719999999999 2024-09-13T09:18:02.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:02.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636612+0000", 2024-09-13T09:18:02.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7499999999999997e-05 2024-09-13T09:18:02.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636669+0000", 2024-09-13T09:18:02.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7286e-05 2024-09-13T09:18:02.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636713+0000", 2024-09-13T09:18:02.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4530000000000002e-05 2024-09-13T09:18:02.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18821 3.4 3:250382e8:::rbd_data.1185c2de398f.0000000000000005:e [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.648454+0000", 2024-09-13T09:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7544058109999998, 2024-09-13T09:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002981521, 2024-09-13T09:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18821 2024-09-13T09:18:02.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.489 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648454+0000", 2024-09-13T09:18:02.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648454+0000", 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648454+0000", 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648455+0000", 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.066e-06 2024-09-13T09:18:02.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648458+0000", 2024-09-13T09:18:02.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3340000000000002e-06 2024-09-13T09:18:02.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648461+0000", 2024-09-13T09:18:02.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-13T09:18:02.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651241+0000", 2024-09-13T09:18:02.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002779625 2024-09-13T09:18:02.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651336+0000", 2024-09-13T09:18:02.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4946999999999997e-05 2024-09-13T09:18:02.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651436+0000", 2024-09-13T09:18:02.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9925000000000003e-05 2024-09-13T09:18:02.493 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.494 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.494 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.494 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18822 3.4 3:250382e8:::rbd_data.1185c2de398f.0000000000000005:e [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:02.494 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:18:02.494 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7543963700000003, 2024-09-13T09:18:02.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031435460000000001, 2024-09-13T09:18:02.494 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.494 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:02.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.495 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.495 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18822 2024-09-13T09:18:02.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.495 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:18:02.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:18:02.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648463+0000", 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-06 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648468+0000", 2024-09-13T09:18:02.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1999999999999999e-06 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648470+0000", 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7389999999999999e-06 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651484+0000", 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003014128 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651541+0000", 2024-09-13T09:18:02.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6304999999999997e-05 2024-09-13T09:18:02.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651607+0000", 2024-09-13T09:18:02.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6280999999999998e-05 2024-09-13T09:18:02.500 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.500 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.500 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.501 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19013 3.4 3:21c7a0ac:::rbd_data.1185b72576e4.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:02.501 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.874194+0000", 2024-09-13T09:18:02.501 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.528665911, 2024-09-13T09:18:02.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033037850000000001, 2024-09-13T09:18:02.501 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.501 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.501 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.501 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.501 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.501 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19013 2024-09-13T09:18:02.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.501 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874194+0000", 2024-09-13T09:18:02.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874194+0000", 2024-09-13T09:18:02.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874191+0000", 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874207+0000", 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6441000000000001e-05 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874235+0000", 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7554000000000001e-05 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874246+0000", 2024-09-13T09:18:02.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1114999999999999e-05 2024-09-13T09:18:02.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874331+0000", 2024-09-13T09:18:02.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4608e-05 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874450+0000", 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011943900000000001 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874607+0000", 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015705600000000001 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:02.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.875492+0000", 2024-09-13T09:18:02.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00088451199999999995 2024-09-13T09:18:02.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:02.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.877353+0000", 2024-09-13T09:18:02.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018616220000000001 2024-09-13T09:18:02.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.877437+0000", 2024-09-13T09:18:02.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3698999999999999e-05 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.877498+0000", 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0745999999999997e-05 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19037 3.4 3:277a4fb8:::rbd_data.1185b72576e4.000000000000015a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.883824+0000", 2024-09-13T09:18:02.507 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.5190356019999998, 2024-09-13T09:18:02.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029514070000000001, 2024-09-13T09:18:02.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.508 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.508 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.508 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.508 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.508 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19037 2024-09-13T09:18:02.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.508 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883824+0000", 2024-09-13T09:18:02.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883824+0000", 2024-09-13T09:18:02.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883822+0000", 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883836+0000", 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3910000000000001e-05 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883891+0000", 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4639999999999999e-05 2024-09-13T09:18:02.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883912+0000", 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1043000000000002e-05 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.884074+0000", 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000162039 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.884308+0000", 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023380800000000001 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.884623+0000", 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031597400000000001 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.886279+0000", 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001655091 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.886674+0000", 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039576299999999999 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.886733+0000", 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8390000000000002e-05 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.886776+0000", 2024-09-13T09:18:02.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2969000000000001e-05 2024-09-13T09:18:02.514 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.514 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.514 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.514 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:20630 3.4 3:23dc35fd:::rbd_data.1185b72576e4.000000000000007c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T09:18:02.514 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:00.986895+0000", 2024-09-13T09:18:02.514 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.4159650319999999, 2024-09-13T09:18:02.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0040477539999999998, 2024-09-13T09:18:02.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 20630 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986895+0000", 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986895+0000", 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986893+0000", 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986899+0000", 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986910+0000", 2024-09-13T09:18:02.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1105e-05 2024-09-13T09:18:02.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986917+0000", 2024-09-13T09:18:02.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:18:02.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986963+0000", 2024-09-13T09:18:02.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5932e-05 2024-09-13T09:18:02.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.987020+0000", 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6858999999999999e-05 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.987104+0000", 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4250000000000001e-05 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.987786+0000", 2024-09-13T09:18:02.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00068249499999999998 2024-09-13T09:18:02.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:02.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.990828+0000", 2024-09-13T09:18:02.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030412099999999999 2024-09-13T09:18:02.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.990893+0000", 2024-09-13T09:18:02.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5117999999999996e-05 2024-09-13T09:18:02.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.990943+0000", 2024-09-13T09:18:02.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9789000000000001e-05 2024-09-13T09:18:02.521 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.521 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.521 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.521 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:21345 3.0 3:004c516c:::rbd_header.1185d6f8ec87:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:18:02.521 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:02.094060+0000", 2024-09-13T09:18:02.521 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.30880005599999999, 2024-09-13T09:18:02.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004611782, 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 21345 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094060+0000", 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094060+0000", 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094057+0000", 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094070+0000", 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2787e-05 2024-09-13T09:18:02.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094095+0000", 2024-09-13T09:18:02.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4873000000000001e-05 2024-09-13T09:18:02.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094106+0000", 2024-09-13T09:18:02.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0954e-05 2024-09-13T09:18:02.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094171+0000", 2024-09-13T09:18:02.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4958999999999999e-05 2024-09-13T09:18:02.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094344+0000", 2024-09-13T09:18:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000173098 2024-09-13T09:18:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094590+0000", 2024-09-13T09:18:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024549199999999999 2024-09-13T09:18:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:02.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.095579+0000", 2024-09-13T09:18:02.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00098965800000000003 2024-09-13T09:18:02.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:02.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.098585+0000", 2024-09-13T09:18:02.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030058390000000002 2024-09-13T09:18:02.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:02.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.098630+0000", 2024-09-13T09:18:02.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5393999999999999e-05 2024-09-13T09:18:02.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:02.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.098672+0000", 2024-09-13T09:18:02.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1153999999999999e-05 2024-09-13T09:18:02.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.528 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.529 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.529 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:02.530 INFO:teuthology.orchestra.run.smithi032.stdout:1857 punch from 0xa77283f to 0xa77912b, (0x68ec bytes) 2024-09-13T09:18:02.530 INFO:teuthology.orchestra.run.smithi032.stdout:1858 read 0xf65577 thru 0xf6a55d (0x4fe7 bytes) 2024-09-13T09:18:02.530 INFO:teuthology.orchestra.run.smithi032.stdout:1862 write 0xadbcb67 thru 0xadbf90b (0x2da5 bytes) 2024-09-13T09:18:02.532 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:02.532 INFO:teuthology.orchestra.run.smithi032.stdout:1863 read 0xacf8f22 thru 0xacf9601 (0x6e0 bytes) 2024-09-13T09:18:02.533 INFO:teuthology.orchestra.run.smithi032.stdout:1864 writesame 0x7c337a1 thru 0x7c3bfdb data_size 0x5d(0x883b bytes) 2024-09-13T09:18:02.533 INFO:teuthology.orchestra.run.smithi032.stdout:1865 writesame 0x265099e thru 0x265c730 data_size 0x7(0xbd93 bytes) 2024-09-13T09:18:02.533 INFO:teuthology.orchestra.run.smithi032.stdout:1866 read 0x3c3e4e9 thru 0x3c4a216 (0xbd2e bytes) 2024-09-13T09:18:02.533 INFO:teuthology.orchestra.run.smithi032.stdout:1869 trunc from 0xd8b1064 to 0xc12b559 2024-09-13T09:18:02.533 INFO:teuthology.orchestra.run.smithi032.stdout:1870 trunc from 0xc12b559 to 0x1f8bcc6 2024-09-13T09:18:02.533 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:02.526+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:02.591 INFO:teuthology.orchestra.run.smithi032.stdout:1871 read 0x20b33c thru 0x20ffdd (0x4ca2 bytes) 2024-09-13T09:18:02.623 INFO:teuthology.orchestra.run.smithi032.stdout:1872 write 0x8931b23 thru 0x8936a07 (0x4ee5 bytes) 2024-09-13T09:18:02.632 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:02.628+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:02.640 INFO:teuthology.orchestra.run.smithi032.stdout:1873 trunc from 0x8936a08 to 0xd94fd73 2024-09-13T09:18:02.651 INFO:teuthology.orchestra.run.smithi032.stdout:1874 punch from 0x298c0cc to 0x299a053, (0xdf87 bytes) 2024-09-13T09:18:02.651 INFO:teuthology.orchestra.run.smithi032.stdout:1875 write 0x1ad7acc thru 0x1adfbe3 (0x8118 bytes) 2024-09-13T09:18:02.654 INFO:teuthology.orchestra.run.smithi032.stdout:1876 punch from 0x17936f1 to 0x179988d, (0x619c bytes) 2024-09-13T09:18:02.674 INFO:teuthology.orchestra.run.smithi032.stdout:1877 trunc from 0xd94fd73 to 0x4fee5b4 2024-09-13T09:18:02.696 INFO:teuthology.orchestra.run.smithi032.stdout:1878 read 0x1042f56 thru 0x1048dd7 (0x5e82 bytes) 2024-09-13T09:18:02.696 INFO:teuthology.orchestra.run.smithi032.stdout:1879 read 0x81986 thru 0x88dae (0x7429 bytes) 2024-09-13T09:18:02.697 INFO:teuthology.orchestra.run.smithi032.stdout:1880 read 0x1e09a5b thru 0x1e124b1 (0x8a57 bytes) 2024-09-13T09:18:02.709 INFO:teuthology.orchestra.run.smithi032.stdout:1881 writesame 0x5123b6f thru 0x512859b data_size 0x1(0x4a2d bytes) 2024-09-13T09:18:02.718 INFO:teuthology.orchestra.run.smithi032.stdout:1882 trunc from 0x512859c to 0x1429193 2024-09-13T09:18:02.719 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:02.719 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:02.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.719 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:21911 3.1e 3:7a6faaa9:::rbd_data.1185d6f8ec87.0000000000000001:head [stat out=16b,truncate 3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:18:02.719 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:02.720135+0000", 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.00074009399999999995, 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00079350500000000001, 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 21911 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:02.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.720135+0000", 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.720135+0000", 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.720132+0000", 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:02.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.720149+0000", 2024-09-13T09:18:02.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6816e-05 2024-09-13T09:18:02.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:02.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.720179+0000", 2024-09-13T09:18:02.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0571999999999998e-05 2024-09-13T09:18:02.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:02.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.720194+0000", 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5238e-05 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.720330+0000", 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013578400000000001 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.720564+0000", 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023425799999999999 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:02.723 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.724 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:02.724 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:18:02.724 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 1 2024-09-13T09:18:02.724 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:02.732 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:02.728+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:02.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:02.741 INFO:teuthology.orchestra.run.smithi032.stdout:1883 trunc from 0x1429193 to 0x5c70ebb 2024-09-13T09:18:02.751 INFO:teuthology.orchestra.run.smithi032.stdout:1884 writesame 0x48ea41 thru 0x49e1fe data_size 0x13(0xf7be bytes) 2024-09-13T09:18:02.769 INFO:teuthology.orchestra.run.smithi032.stdout:1886 write 0x8678ebc thru 0x868602c (0xd171 bytes) 2024-09-13T09:18:02.776 INFO:teuthology.orchestra.run.smithi032.stdout:1887 read 0x4536a0f thru 0x453f1e2 (0x87d4 bytes) 2024-09-13T09:18:02.787 INFO:teuthology.orchestra.run.smithi032.stdout:1889 trunc from 0x868602d to 0x15376d6 2024-09-13T09:18:02.809 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:02.809 INFO:teuthology.orchestra.run.smithi032.stdout:1892 trunc from 0x15376d6 to 0x1aaaa6f 2024-09-13T09:18:02.828 INFO:teuthology.orchestra.run.smithi032.stdout:1896 trunc from 0x1aaaa6f to 0x7eb21ff 2024-09-13T09:18:02.833 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:02.832+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:02.837 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:02.837 INFO:teuthology.orchestra.run.smithi032.stdout:1898 read 0x69b39a5 thru 0x69bc75a (0x8db6 bytes) 2024-09-13T09:18:02.843 INFO:teuthology.orchestra.run.smithi032.stdout:1899 trunc from 0x7eb21ff to 0xbc4bbe3 2024-09-13T09:18:02.865 INFO:teuthology.orchestra.run.smithi032.stdout:1902 writesame 0xda42ece thru 0xda52706 data_size 0x47(0xf839 bytes) 2024-09-13T09:18:02.865 INFO:teuthology.orchestra.run.smithi032.stdout:1903 read 0xd77c3c6 thru 0xd77f986 (0x35c1 bytes) 2024-09-13T09:18:02.865 INFO:teuthology.orchestra.run.smithi032.stdout:1904 writesame 0xab86854 thru 0xab910f8 data_size 0x75(0xa8a5 bytes) 2024-09-13T09:18:02.873 INFO:teuthology.orchestra.run.smithi032.stdout:1905 writesame 0xb0738b7 thru 0xb07cc49 data_size 0x31(0x9393 bytes) 2024-09-13T09:18:02.879 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:02.879 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:02.900 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:02.907 INFO:teuthology.orchestra.run.smithi032.stdout:1906 trunc from 0xda52707 to 0x2ea93f 2024-09-13T09:18:02.955 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:18:02.955 INFO:teuthology.orchestra.run.smithi032.stderr:marked in osd.1. 2024-09-13T09:18:02.957 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:02.952+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:02.982 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-13T09:18:03.035 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:03.032+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:03.035 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:18:03.045 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.509054426000001, 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:18:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:18:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:18:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:18:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:18:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:18:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:18:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19325 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2514944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1155777599999999, 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041428072000000003, 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19325 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929839+0000", 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929845+0000", 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9510000000000003e-06 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929850+0000", 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971169+0000", 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041318743999999998 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971214+0000", 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:18:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971265+0000", 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0963e-05 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19326 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2613248~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1155608240000001, 2024-09-13T09:18:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041534459000000003, 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19326 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929856+0000", 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0470000000000001e-06 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929862+0000", 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3300000000000004e-06 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929867+0000", 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971293+0000", 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041425785999999999 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971337+0000", 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4240999999999999e-05 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971389+0000", 2024-09-13T09:18:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1483999999999997e-05 2024-09-13T09:18:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19327 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2711552~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1155437580000003, 2024-09-13T09:18:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041642875000000003, 2024-09-13T09:18:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19327 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929873+0000", 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.26e-06 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929879+0000", 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0519999999999998e-06 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929884+0000", 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971416+0000", 2024-09-13T09:18:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041532122999999997 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971460+0000", 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4456999999999998e-05 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971514+0000", 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3579000000000003e-05 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19328 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2809856~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1155269949999997, 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041750662000000001, 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19328 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929890+0000", 2024-09-13T09:18:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:18:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929896+0000", 2024-09-13T09:18:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:18:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929901+0000", 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971542+0000", 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041640442 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971587+0000", 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4972999999999998e-05 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971639+0000", 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1990000000000002e-05 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19329 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2908160~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1154850029999999, 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041833958999999997, 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19329 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:18:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:18:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929929+0000", 2024-09-13T09:18:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929932+0000", 2024-09-13T09:18:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0699999999999998e-06 2024-09-13T09:18:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929941+0000", 2024-09-13T09:18:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.149e-06 2024-09-13T09:18:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929947+0000", 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.699e-06 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971666+0000", 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041719224999999999 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971711+0000", 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4823000000000001e-05 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971764+0000", 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2741999999999997e-05 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19330 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 3006464~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1154622620000003, 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041949980999999997, 2024-09-13T09:18:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19330 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:18:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929952+0000", 2024-09-13T09:18:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929955+0000", 2024-09-13T09:18:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:18:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929961+0000", 2024-09-13T09:18:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1199999999999999e-06 2024-09-13T09:18:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929966+0000", 2024-09-13T09:18:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:18:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971792+0000", 2024-09-13T09:18:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041826680999999998 2024-09-13T09:18:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971838+0000", 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5939000000000002e-05 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971903+0000", 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4499999999999996e-05 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19331 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 3104768~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1154452949999998, 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042058848000000003, 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19331 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:18:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:18:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929969+0000", 2024-09-13T09:18:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929975+0000", 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.434e-06 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929981+0000", 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929986+0000", 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1780000000000002e-06 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971931+0000", 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041944417999999997 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971976+0000", 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4962000000000001e-05 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972029+0000", 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2768e-05 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19332 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 65536~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1154181249999997, 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042158568, 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19332 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929996+0000", 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929999+0000", 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.306e-06 2024-09-13T09:18:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930005+0000", 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0379999999999996e-06 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930010+0000", 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0969999999999996e-06 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972057+0000", 2024-09-13T09:18:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042047117000000002 2024-09-13T09:18:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972102+0000", 2024-09-13T09:18:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5422e-05 2024-09-13T09:18:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972155+0000", 2024-09-13T09:18:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2964999999999999e-05 2024-09-13T09:18:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19333 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 163840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1154010349999997, 2024-09-13T09:18:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042266805999999997, 2024-09-13T09:18:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19333 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930016+0000", 2024-09-13T09:18:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:18:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930022+0000", 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2550000000000003e-06 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930027+0000", 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8060000000000002e-06 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972183+0000", 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042156276999999999 2024-09-13T09:18:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972228+0000", 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5317e-05 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972281+0000", 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2398999999999998e-05 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19334 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 262144~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:18:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1153831900000002, 2024-09-13T09:18:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042375573, 2024-09-13T09:18:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19334 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930031+0000", 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930034+0000", 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930040+0000", 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0789999999999997e-06 2024-09-13T09:18:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930045+0000", 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0139999999999998e-06 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972307+0000", 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042262685000000001 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972352+0000", 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5021000000000001e-05 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972407+0000", 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4945000000000003e-05 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19335 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 360448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1153663759999999, 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042483785000000003, 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19335 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930048+0000", 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930050+0000", 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.136e-06 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930056+0000", 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930061+0000", 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9289999999999997e-06 2024-09-13T09:18:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972435+0000", 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042373347999999998 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972479+0000", 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4597e-05 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972532+0000", 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3034000000000001e-05 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19336 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 458752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1153494390000001, 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042590488000000003, 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19336 2024-09-13T09:18:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:18:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930065+0000", 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930067+0000", 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2110000000000001e-06 2024-09-13T09:18:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930074+0000", 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4640000000000003e-06 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930079+0000", 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7709999999999997e-06 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972560+0000", 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042481738999999998 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972604+0000", 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4060999999999997e-05 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972656+0000", 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.164e-05 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19337 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1153322640000001, 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042697997000000001, 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19337 2024-09-13T09:18:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:18:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:18:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930082+0000", 2024-09-13T09:18:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930085+0000", 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.221e-06 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930091+0000", 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.942e-06 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930096+0000", 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.04e-06 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972683+0000", 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042587885999999998 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972728+0000", 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4058999999999999e-05 2024-09-13T09:18:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972781+0000", 2024-09-13T09:18:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3156999999999999e-05 2024-09-13T09:18:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19342 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1048576~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1134148369999997, 2024-09-13T09:18:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041418260999999998, 2024-09-13T09:18:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19342 2024-09-13T09:18:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932002+0000", 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3180000000000001e-06 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932008+0000", 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2199999999999997e-06 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932013+0000", 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:18:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973319+0000", 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041305738000000002 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973364+0000", 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5809000000000001e-05 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973418+0000", 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3888000000000003e-05 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19343 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1146880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1133876300000001, 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041515107000000002, 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19343 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932034+0000", 2024-09-13T09:18:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-13T09:18:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932040+0000", 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2260000000000002e-06 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932045+0000", 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5589999999999999e-06 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973446+0000", 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041401070999999998 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973491+0000", 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5021999999999997e-05 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973542+0000", 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1465000000000003e-05 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19344 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:18:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1133651039999997, 2024-09-13T09:18:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041617848999999998, 2024-09-13T09:18:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19344 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932049+0000", 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932052+0000", 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4540000000000001e-06 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932064+0000", 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1911e-05 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932079+0000", 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4765e-05 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973570+0000", 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041491800000000002 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973614+0000", 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4149e-05 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973668+0000", 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3198999999999997e-05 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19345 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1343488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.113328933, 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041707699000000001, 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19345 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:18:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:18:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932085+0000", 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932089+0000", 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7009999999999998e-06 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932097+0000", 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3010000000000007e-06 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932104+0000", 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9260000000000003e-06 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973695+0000", 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041591056000000001 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973741+0000", 2024-09-13T09:18:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6003e-05 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973794+0000", 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2265999999999997e-05 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19346 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1441792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1133048319999999, 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041808933999999999, 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19346 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932112+0000", 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932119+0000", 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8369999999999999e-06 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932125+0000", 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973822+0000", 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041697369999999997 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973867+0000", 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4648999999999999e-05 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973919+0000", 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2451000000000003e-05 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19347 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1540096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.1132853789999997, 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041915045999999997, 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19347 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:18:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932129+0000", 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932131+0000", 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.278e-06 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932139+0000", 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.43e-06 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932145+0000", 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7939999999999999e-06 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973946+0000", 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041801724999999998 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973993+0000", 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6245000000000002e-05 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.974045+0000", 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1950999999999998e-05 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:03.146 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:03.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:03.148 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:03.142+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:03.172 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:03.172 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:03.188 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:03.204 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:03.220 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:03.220 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:03.239 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:03.240 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:03.248 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:03.244+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:03.258 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:03.278 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:03.284 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:03.285 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:03.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:03.299 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:03.362 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:03.360+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:03.418 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:03.418 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:03.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:03.478 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:03.474+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:03.550 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:03.546+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:03.556 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:03.557 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:03.571 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:03.652 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:03.652+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:03.739 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:03.739 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:03.753 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:03.754 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:03.746+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:03.853 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:03.850+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:03.927 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:03.927 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:03.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:03.962 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:03.960+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:03.991 INFO:teuthology.orchestra.run.smithi032.stdout:1907 writesame 0xe0aea97 thru 0xe0ba4bc data_size 0x1(0xba26 bytes) 2024-09-13T09:18:03.991 INFO:teuthology.orchestra.run.smithi032.stdout:1909 writesame 0x53dadb7 thru 0x53df02f data_size 0xd(0x4279 bytes) 2024-09-13T09:18:04.001 INFO:teuthology.orchestra.run.smithi032.stdout:1911 trunc from 0xe0ba4bd to 0xcf030fa 2024-09-13T09:18:04.016 INFO:teuthology.orchestra.run.smithi032.stdout:1912 writesame 0xbca1ef8 thru 0xbca6b87 data_size 0x5(0x4c90 bytes) 2024-09-13T09:18:04.023 INFO:teuthology.orchestra.run.smithi032.stdout:1913 read 0x658f056 thru 0x6598f9b (0x9f46 bytes) 2024-09-13T09:18:04.023 INFO:teuthology.orchestra.run.smithi032.stdout:1914 read 0x1694553 thru 0x169ece4 (0xa792 bytes) 2024-09-13T09:18:04.024 INFO:teuthology.orchestra.run.smithi032.stdout:1917 read 0x856aea3 thru 0x856c018 (0x1176 bytes) 2024-09-13T09:18:04.024 INFO:teuthology.orchestra.run.smithi032.stdout:1918 read 0xb07a68d thru 0xb07bb35 (0x14a9 bytes) 2024-09-13T09:18:04.024 INFO:teuthology.orchestra.run.smithi032.stdout:1920 punch from 0x425870a to 0x4264545, (0xbe3b bytes) 2024-09-13T09:18:04.024 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:04.024 INFO:teuthology.orchestra.run.smithi032.stdout:1924 write 0x6797a66 thru 0x67a6459 (0xe9f4 bytes) 2024-09-13T09:18:04.030 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:04.030 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:04.030 INFO:teuthology.orchestra.run.smithi032.stdout:1927 read 0x4dd403e thru 0x4dd97f7 (0x57ba bytes) 2024-09-13T09:18:04.056 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:04.052+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:04.059 INFO:teuthology.orchestra.run.smithi032.stdout:1930 trunc from 0xcf030fa to 0x34db1e1 2024-09-13T09:18:04.110 INFO:teuthology.orchestra.run.smithi032.stdout:1931 write 0x7f06f0f thru 0x7f0f23f (0x8331 bytes) 2024-09-13T09:18:04.116 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:04.126 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:04.126 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:04.128 INFO:teuthology.orchestra.run.smithi032.stdout:1934 trunc from 0x7f0f240 to 0xd8b9dc0 2024-09-13T09:18:04.138 INFO:teuthology.orchestra.run.smithi032.stdout:1935 read 0x2040355 thru 0x2046930 (0x65dc bytes) 2024-09-13T09:18:04.138 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:04.139 INFO:teuthology.orchestra.run.smithi032.stdout:1937 punch from 0x98dce11 to 0x98e4477, (0x7666 bytes) 2024-09-13T09:18:04.139 INFO:teuthology.orchestra.run.smithi032.stdout:1938 punch from 0xceb4a2a to 0xceba578, (0x5b4e bytes) 2024-09-13T09:18:04.139 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:04.139 INFO:teuthology.orchestra.run.smithi032.stdout:1941 read 0xcf931e1 thru 0xcf9da60 (0xa880 bytes) 2024-09-13T09:18:04.141 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:04.165 INFO:teuthology.orchestra.run.smithi032.stdout:1945 trunc from 0xd8b9dc0 to 0x1804e9 2024-09-13T09:18:04.185 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:04.182+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:04.203 INFO:teuthology.orchestra.run.smithi032.stdout:1947 write 0x3e05101 thru 0x3e0c794 (0x7694 bytes) 2024-09-13T09:18:04.221 INFO:teuthology.orchestra.run.smithi032.stdout:1949 trunc from 0x3e0c795 to 0xafccbff 2024-09-13T09:18:04.229 INFO:teuthology.orchestra.run.smithi032.stdout:1950 punch from 0x7887dd3 to 0x789148e, (0x96bb bytes) 2024-09-13T09:18:04.229 INFO:teuthology.orchestra.run.smithi032.stdout:1951 punch from 0x1b71c60 to 0x1b815ab, (0xf94b bytes) 2024-09-13T09:18:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:1952 write 0xdf124af thru 0xdf169d5 (0x4527 bytes) 2024-09-13T09:18:04.249 INFO:teuthology.orchestra.run.smithi032.stdout:1953 read 0xc4a73f5 thru 0xc4ab461 (0x406d bytes) 2024-09-13T09:18:04.249 INFO:teuthology.orchestra.run.smithi032.stdout:1954 punch from 0x742da2 to 0x751ddf, (0xf03d bytes) 2024-09-13T09:18:04.249 INFO:teuthology.orchestra.run.smithi032.stdout:1955 punch from 0xbabd797 to 0xbacc0f7, (0xe960 bytes) 2024-09-13T09:18:04.249 INFO:teuthology.orchestra.run.smithi032.stdout:1956 write 0xce911a6 thru 0xce95bf8 (0x4a53 bytes) 2024-09-13T09:18:04.255 INFO:teuthology.orchestra.run.smithi032.stdout:1957 punch from 0x2cafc79 to 0x2cbf240, (0xf5c7 bytes) 2024-09-13T09:18:04.256 INFO:teuthology.orchestra.run.smithi032.stdout:1958 write 0x82e2866 thru 0x82ef7e1 (0xcf7c bytes) 2024-09-13T09:18:04.258 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:04.254+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:04.261 INFO:teuthology.orchestra.run.smithi032.stdout:1959 read 0xb5fe1bf thru 0xb6038ea (0x572c bytes) 2024-09-13T09:18:04.262 INFO:teuthology.orchestra.run.smithi032.stdout:1960 write 0xce1874d thru 0xce25c02 (0xd4b6 bytes) 2024-09-13T09:18:04.269 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:04.269 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:04.269 INFO:teuthology.orchestra.run.smithi032.stdout:1961 write 0x2cf1b1c thru 0x2cf3555 (0x1a3a bytes) 2024-09-13T09:18:04.274 INFO:teuthology.orchestra.run.smithi032.stdout:1962 write 0x85eb9c3 thru 0x85f29b0 (0x6fee bytes) 2024-09-13T09:18:04.280 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:04.280 INFO:teuthology.orchestra.run.smithi032.stdout:1965 read 0x15bfcdc thru 0x15c3ca4 (0x3fc9 bytes) 2024-09-13T09:18:04.280 INFO:teuthology.orchestra.run.smithi032.stdout:1966 writesame 0xad32c4e thru 0xad38354 data_size 0x1(0x5707 bytes) 2024-09-13T09:18:04.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:04.374 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:04.370+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:04.453 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:04.453 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:04.459 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:04.460+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:04.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:04.560 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:04.556+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:04.646 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:04.661 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:04.656+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:04.762 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:04.760+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:04.774 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:04.774 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:04.788 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:04.863 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:04.860+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:04.908 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:04.908 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:04.922 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:04.970 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:04.966+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:04.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:18:05.022 INFO:teuthology.orchestra.run.smithi032.stdout:1968 trunc from 0xdf169d6 to 0xac4e862 2024-09-13T09:18:05.058 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:05.058 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:05.059 INFO:teuthology.orchestra.run.smithi032.stdout:1969 trunc from 0xac4e862 to 0x4f9c0d 2024-09-13T09:18:05.065 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:05.058+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:05.077 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:05.084 INFO:teuthology.orchestra.run.smithi032.stdout:1972 trunc from 0x4f9c0d to 0x2934af4 2024-09-13T09:18:05.092 INFO:teuthology.orchestra.run.smithi032.stdout:1973 trunc from 0x2934af4 to 0x26bb83c 2024-09-13T09:18:05.129 INFO:teuthology.orchestra.run.smithi032.stdout:1974 write 0xdef0b4d thru 0xdefc152 (0xb606 bytes) 2024-09-13T09:18:05.135 INFO:teuthology.orchestra.run.smithi032.stdout:1975 read 0xad9ae3c thru 0xada6039 (0xb1fe bytes) 2024-09-13T09:18:05.136 INFO:teuthology.orchestra.run.smithi032.stdout:1976 write 0x5761fe7 thru 0x57667f8 (0x4812 bytes) 2024-09-13T09:18:05.141 INFO:teuthology.orchestra.run.smithi032.stdout:1979 write 0x7756058 thru 0x775daab (0x7a54 bytes) 2024-09-13T09:18:05.145 INFO:teuthology.orchestra.run.smithi032.stdout:1981 read 0x76c4bc thru 0x77a5b0 (0xe0f5 bytes) 2024-09-13T09:18:05.147 INFO:teuthology.orchestra.run.smithi032.stdout:1982 read 0xdabe6f7 thru 0xdac28d0 (0x41da bytes) 2024-09-13T09:18:05.147 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:05.147 INFO:teuthology.orchestra.run.smithi032.stdout:1984 read 0x82b70f1 thru 0x82b9908 (0x2818 bytes) 2024-09-13T09:18:05.147 INFO:teuthology.orchestra.run.smithi032.stdout:1985 punch from 0x8b95246 to 0x8ba0721, (0xb4db bytes) 2024-09-13T09:18:05.147 INFO:teuthology.orchestra.run.smithi032.stdout:1987 read 0xac5a59a thru 0xac61307 (0x6d6e bytes) 2024-09-13T09:18:05.166 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:05.164+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:05.166 INFO:teuthology.orchestra.run.smithi032.stdout:1989 trunc from 0xdefc153 to 0x36afeb 2024-09-13T09:18:05.204 INFO:teuthology.orchestra.run.smithi032.stdout:1990 trunc from 0x36afeb to 0xc8b40d8 2024-09-13T09:18:05.213 INFO:teuthology.orchestra.run.smithi032.stdout:1991 read 0x5e5379c thru 0x5e5b2ab (0x7b10 bytes) 2024-09-13T09:18:05.213 INFO:teuthology.orchestra.run.smithi032.stdout:1992 trunc from 0xc8b40d8 to 0xd13bd5c 2024-09-13T09:18:05.231 INFO:teuthology.orchestra.run.smithi032.stdout:1993 writesame 0xd7b5462 thru 0xd7bd761 data_size 0x40(0x8300 bytes) 2024-09-13T09:18:05.231 INFO:teuthology.orchestra.run.smithi032.stdout:1994 write 0x507236d thru 0x5075bee (0x3882 bytes) 2024-09-13T09:18:05.246 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:05.246 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:05.247 INFO:teuthology.orchestra.run.smithi032.stdout:1995 write 0xdb54dde thru 0xdb5bcda (0x6efd bytes) 2024-09-13T09:18:05.255 INFO:teuthology.orchestra.run.smithi032.stdout:1996 writesame 0x3c92d25 thru 0x3ca2436 data_size 0x7d(0xf712 bytes) 2024-09-13T09:18:05.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:05.262 INFO:teuthology.orchestra.run.smithi032.stdout:1997 read 0x779b743 thru 0x77a4eb5 (0x9773 bytes) 2024-09-13T09:18:05.275 INFO:teuthology.orchestra.run.smithi032.stdout:1998 trunc from 0xdb5bcdb to 0x4760d5c 2024-09-13T09:18:05.276 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:05.272+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:05.312 INFO:teuthology.orchestra.run.smithi032.stdout:1999 writesame 0x88a96a8 thru 0x88ae449 data_size 0x13(0x4da2 bytes) 2024-09-13T09:18:05.312 INFO:teuthology.orchestra.run.smithi032.stdout:2001 read 0x1d8c81f thru 0x1d8f85d (0x303f bytes) 2024-09-13T09:18:05.313 INFO:teuthology.orchestra.run.smithi032.stdout:2002 read 0x7302a8b thru 0x730a034 (0x75aa bytes) 2024-09-13T09:18:05.319 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:05.319 INFO:teuthology.orchestra.run.smithi032.stdout:2003 trunc from 0x88ae44a to 0x45e5890 2024-09-13T09:18:05.331 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:05.348 INFO:teuthology.orchestra.run.smithi032.stdout:2006 trunc from 0x45e5890 to 0xdd30094 2024-09-13T09:18:05.368 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:05.364+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:05.372 INFO:teuthology.orchestra.run.smithi032.stdout:2008 trunc from 0xdd30094 to 0x4a12f3e 2024-09-13T09:18:05.385 INFO:teuthology.orchestra.run.smithi032.stdout:2009 read 0x2c11201 thru 0x2c1c858 (0xb658 bytes) 2024-09-13T09:18:05.385 INFO:teuthology.orchestra.run.smithi032.stdout:2014 read 0x14deca1 thru 0x14e05de (0x193e bytes) 2024-09-13T09:18:05.385 INFO:teuthology.orchestra.run.smithi032.stdout:2017 read 0x33f7f0a thru 0x34024f3 (0xa5ea bytes) 2024-09-13T09:18:05.401 INFO:teuthology.orchestra.run.smithi032.stdout:2019 writesame 0xa02216a thru 0xa028ed6 data_size 0x1(0x6d6d bytes) 2024-09-13T09:18:05.402 INFO:teuthology.orchestra.run.smithi032.stdout:2021 read 0x5028f9d thru 0x502c761 (0x37c5 bytes) 2024-09-13T09:18:05.402 INFO:teuthology.orchestra.run.smithi032.stdout:2022 read 0x756ee0c thru 0x757a0d8 (0xb2cd bytes) 2024-09-13T09:18:05.402 INFO:teuthology.orchestra.run.smithi032.stdout:2023 write 0x2ed0fb9 thru 0x2edf761 (0xe7a9 bytes) 2024-09-13T09:18:05.406 INFO:teuthology.orchestra.run.smithi032.stdout:2024 write 0x1c1ba3d thru 0x1c21e69 (0x642d bytes) 2024-09-13T09:18:05.408 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:05.408 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:05.408 INFO:teuthology.orchestra.run.smithi032.stdout:2025 punch from 0x1c854e1 to 0x1c8e03e, (0x8b5d bytes) 2024-09-13T09:18:05.422 INFO:teuthology.orchestra.run.smithi032.stdout:2026 write 0xce577cc thru 0xce5bac9 (0x42fe bytes) 2024-09-13T09:18:05.428 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:05.430 INFO:teuthology.orchestra.run.smithi032.stdout:2027 read 0x27a9303 thru 0x27b4d4c (0xba4a bytes) 2024-09-13T09:18:05.432 INFO:teuthology.orchestra.run.smithi032.stdout:2028 write 0xa672994 thru 0xa674874 (0x1ee1 bytes) 2024-09-13T09:18:05.446 INFO:teuthology.orchestra.run.smithi032.stdout:2030 trunc from 0xce5baca to 0x6511376 2024-09-13T09:18:05.468 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:05.462+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:05.479 INFO:teuthology.orchestra.run.smithi032.stdout:2031 trunc from 0x6511376 to 0xd6601be 2024-09-13T09:18:05.485 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:05.485 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:05.488 INFO:teuthology.orchestra.run.smithi032.stdout:2032 trunc from 0xd6601be to 0xca3d847 2024-09-13T09:18:05.496 INFO:teuthology.orchestra.run.smithi032.stdout:2034 read 0x3321d2b thru 0x3322ae7 (0xdbd bytes) 2024-09-13T09:18:05.496 INFO:teuthology.orchestra.run.smithi032.stdout:2035 punch from 0xac450b2 to 0xac47509, (0x2457 bytes) 2024-09-13T09:18:05.496 INFO:teuthology.orchestra.run.smithi032.stdout:2036 writesame 0x16fef3b thru 0x17051f5 data_size 0x19(0x62bb bytes) 2024-09-13T09:18:05.503 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:05.504 INFO:teuthology.orchestra.run.smithi032.stdout:2038 writesame 0xbc55e14 thru 0xbc563d9 data_size 0x1(0x5c6 bytes) 2024-09-13T09:18:05.510 INFO:teuthology.orchestra.run.smithi032.stdout:2040 write 0xa873dc3 thru 0xa87e7a8 (0xa9e6 bytes) 2024-09-13T09:18:05.517 INFO:teuthology.orchestra.run.smithi032.stdout:2041 writesame 0x773dca8 thru 0x774b34b data_size 0x72(0xd6a4 bytes) 2024-09-13T09:18:05.520 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:05.524 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:05.538 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:05.539 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:05.540 INFO:teuthology.orchestra.run.smithi032.stdout:2043 trunc from 0xca3d847 to 0xc485cd 2024-09-13T09:18:05.561 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:05.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3:ed699f0f:::rbd_data.118591b29fad.000000000000008c:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:18:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:18:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.699751829, 2024-09-13T09:18:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94878089700000001, 2024-09-13T09:18:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.562 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:18:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:18:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:18:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:18:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:18:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:18:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810654+0000", 2024-09-13T09:18:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94834837599999999 2024-09-13T09:18:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810755+0000", 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100987 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810828+0000", 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2751999999999997e-05 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4104 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.567 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.588274071000001, 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023862547000000001, 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4104 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973525+0000", 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973524+0000", 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973527+0000", 2024-09-13T09:18:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1200000000000002e-06 2024-09-13T09:18:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973535+0000", 2024-09-13T09:18:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2479999999999999e-06 2024-09-13T09:18:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973548+0000", 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.365e-05 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997205+0000", 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023657088999999999 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997289+0000", 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3801000000000006e-05 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997387+0000", 2024-09-13T09:18:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8317000000000001e-05 2024-09-13T09:18:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4105 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:18:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.588245554, 2024-09-13T09:18:05.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024049968000000001, 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4105 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973553+0000", 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973556+0000", 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7420000000000002e-06 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973564+0000", 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973569+0000", 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997418+0000", 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023849052999999999 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997504+0000", 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6075999999999994e-05 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997603+0000", 2024-09-13T09:18:05.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9455000000000002e-05 2024-09-13T09:18:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4106 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:18:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.588225962999999, 2024-09-13T09:18:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024364336, 2024-09-13T09:18:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4106 2024-09-13T09:18:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:18:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973573+0000", 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973575+0000", 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.322e-06 2024-09-13T09:18:05.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973594+0000", 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9031000000000001e-05 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973601+0000", 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5300000000000002e-06 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997648+0000", 2024-09-13T09:18:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024047731999999999 2024-09-13T09:18:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997748+0000", 2024-09-13T09:18:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9807000000000002e-05 2024-09-13T09:18:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997937+0000", 2024-09-13T09:18:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018924299999999999 2024-09-13T09:18:05.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4107 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.588193519000001, 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542690999999998, 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.582 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4107 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973605+0000", 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973614+0000", 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973621+0000", 2024-09-13T09:18:05.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0110000000000003e-06 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973626+0000", 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0320000000000003e-06 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.997979+0000", 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024353063000000001 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998059+0000", 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9221000000000006e-05 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998148+0000", 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9506999999999995e-05 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4108 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:18:05.586 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.588167910999999, 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024717223999999999, 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4108 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:18:05.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973631+0000", 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973633+0000", 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2089999999999999e-06 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973639+0000", 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1469999999999998e-06 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973644+0000", 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5310000000000003e-06 2024-09-13T09:18:05.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998175+0000", 2024-09-13T09:18:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024531826999999999 2024-09-13T09:18:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998257+0000", 2024-09-13T09:18:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1668999999999993e-05 2024-09-13T09:18:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998348+0000", 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1184e-05 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4109 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.58814838, 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024935052999999999, 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.591 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4109 2024-09-13T09:18:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:18:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973651+0000", 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973650+0000", 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973653+0000", 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.328e-06 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973659+0000", 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3269999999999997e-06 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973664+0000", 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8149999999999996e-06 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998375+0000", 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024711767999999999 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998484+0000", 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108257 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998586+0000", 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101953 2024-09-13T09:18:05.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4110 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.588131045000001, 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025120632, 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4110 2024-09-13T09:18:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:18:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:18:05.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973668+0000", 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973678+0000", 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0712e-05 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973685+0000", 2024-09-13T09:18:05.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9029999999999998e-06 2024-09-13T09:18:05.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973690+0000", 2024-09-13T09:18:05.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.955e-06 2024-09-13T09:18:05.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998613+0000", 2024-09-13T09:18:05.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024922546 2024-09-13T09:18:05.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998695+0000", 2024-09-13T09:18:05.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2066999999999999e-05 2024-09-13T09:18:05.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998789+0000", 2024-09-13T09:18:05.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3813000000000005e-05 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4111 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.588104207000001, 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025376396999999998, 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4111 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973695+0000", 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973694+0000", 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973697+0000", 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.384e-06 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973703+0000", 2024-09-13T09:18:05.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9970000000000004e-06 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973707+0000", 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7169999999999999e-06 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998817+0000", 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025109446000000001 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.998898+0000", 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1003999999999999e-05 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999071+0000", 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000173337 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.605 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4112 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.588087383000001, 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025596280999999999, 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4112 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973712+0000", 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973711+0000", 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973713+0000", 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2670000000000001e-06 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973719+0000", 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9379999999999997e-06 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973724+0000", 2024-09-13T09:18:05.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6809999999999997e-06 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999115+0000", 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025390511000000001 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999202+0000", 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7787999999999994e-05 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999308+0000", 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105568 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4113 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 430080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.588070238, 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025814090000000001, 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.610 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4113 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973729+0000", 2024-09-13T09:18:05.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973728+0000", 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973738+0000", 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4339999999999998e-06 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973745+0000", 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9639999999999997e-06 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973754+0000", 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4760000000000004e-06 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999360+0000", 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025606035999999999 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999443+0000", 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3393000000000006e-05 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999543+0000", 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9351000000000005e-05 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.614 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4114 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.588040382999999, 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026036354000000001, 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4114 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:18:05.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973759+0000", 2024-09-13T09:18:05.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973758+0000", 2024-09-13T09:18:05.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973761+0000", 2024-09-13T09:18:05.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-13T09:18:05.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973767+0000", 2024-09-13T09:18:05.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6209999999999998e-06 2024-09-13T09:18:05.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973772+0000", 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999593+0000", 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025820532 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999691+0000", 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8701000000000002e-05 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999795+0000", 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103467 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4115 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.619 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.588022639999998, 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026260549000000001, 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4115 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:18:05.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:18:05.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973776+0000", 2024-09-13T09:18:05.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973778+0000", 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.091e-06 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973784+0000", 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.268e-06 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973789+0000", 2024-09-13T09:18:05.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4599999999999996e-06 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999836+0000", 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026046980000000001 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.999932+0000", 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5863e-05 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000037+0000", 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105264 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4116 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.588006252, 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026488986999999999, 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.624 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4116 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973793+0000", 2024-09-13T09:18:05.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973792+0000", 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973794+0000", 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8950000000000001e-06 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973800+0000", 2024-09-13T09:18:05.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9780000000000002e-06 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973811+0000", 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1192999999999999e-05 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000081+0000", 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026269942000000001 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000175+0000", 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3819000000000004e-05 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000282+0000", 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106507 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4117 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.628 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:18:05.629 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.587982707999998, 2024-09-13T09:18:05.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026706133999999999, 2024-09-13T09:18:05.629 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.629 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.629 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.629 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.629 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4117 2024-09-13T09:18:05.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.629 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973816+0000", 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973818+0000", 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6259999999999999e-06 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973825+0000", 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.737e-06 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973830+0000", 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4399999999999998e-06 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000317+0000", 2024-09-13T09:18:05.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026486948 2024-09-13T09:18:05.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000398+0000", 2024-09-13T09:18:05.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1160000000000002e-05 2024-09-13T09:18:05.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000522+0000", 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012466800000000001 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4118 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.587965179000001, 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026891327, 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.633 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4118 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973834+0000", 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973833+0000", 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973836+0000", 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2350000000000002e-06 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973841+0000", 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7230000000000001e-06 2024-09-13T09:18:05.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973846+0000", 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1829999999999996e-06 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000563+0000", 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026717797000000001 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000630+0000", 2024-09-13T09:18:05.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6152e-05 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000725+0000", 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5601999999999997e-05 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:4119 3.1b 3:deeb9fd3:::rbd_data.1185be6cd1eb.0000000000000009:6 [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.587949241, 2024-09-13T09:18:05.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027064775999999999, 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 4119 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973850+0000", 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973849+0000", 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973852+0000", 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973859+0000", 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7589999999999997e-06 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.973864+0000", 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8119999999999998e-06 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000747+0000", 2024-09-13T09:18:05.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026882653999999999 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000816+0000", 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8841e-05 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.000915+0000", 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8933999999999998e-05 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14583 3.1f 3:feba6951:::rbd_data.11853ecd5f5f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.027768564999999, 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.050883949999999997, 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14583 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.642 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:18:05.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534030+0000", 2024-09-13T09:18:05.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.534028+0000", 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538304+0000", 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00427634 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538342+0000", 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7567000000000001e-05 2024-09-13T09:18:05.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538357+0000", 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4936999999999999e-05 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538411+0000", 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4166000000000003e-05 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.538539+0000", 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000127966 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.539016+0000", 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477014 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.547890+0000", 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0088736430000000005 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584825+0000", 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036934966 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584866+0000", 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1514000000000003e-05 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.584914+0000", 2024-09-13T09:18:05.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8167000000000001e-05 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16399 3.1f 3:fcbf421e:::rbd_data.1185f20bdb42.0000000000000019:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.655128+0000", 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.906670910999999, 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042135120999999998, 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.648 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16399 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655128+0000", 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655128+0000", 2024-09-13T09:18:05.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655126+0000", 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655139+0000", 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3373000000000001e-05 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655164+0000", 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4576999999999999e-05 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655177+0000", 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2853e-05 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655257+0000", 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0445999999999995e-05 2024-09-13T09:18:05.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.655382+0000", 2024-09-13T09:18:05.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012448900000000001 2024-09-13T09:18:05.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:05.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.663115+0000", 2024-09-13T09:18:05.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077335030000000001 2024-09-13T09:18:05.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.697103+0000", 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033987512999999997 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.697165+0000", 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2731000000000006e-05 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.697223+0000", 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7651000000000001e-05 2024-09-13T09:18:05.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.697263+0000", 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0061999999999997e-05 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19465 3.10 3:0e3a5b30:::rbd_data.1185b72576e4.0000000000000062:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6139070919999998, 2024-09-13T09:18:05.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97489159000000003, 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19465 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947890+0000", 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947899+0000", 2024-09-13T09:18:05.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6840000000000008e-06 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947917+0000", 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7731e-05 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947925+0000", 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4189999999999996e-06 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947990+0000", 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5909999999999997e-05 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947998+0000", 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4449999999999999e-06 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.919994+0000", 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97199595900000002 2024-09-13T09:18:05.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920258+0000", 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026439500000000001 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920629+0000", 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037096299999999998 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:05.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.921797+0000", 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001167721 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922544+0000", 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00074661200000000001 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922628+0000", 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4016000000000003e-05 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:05.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:05.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:05.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922783+0000", 2024-09-13T09:18:05.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015585499999999999 2024-09-13T09:18:05.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.661 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:05.661 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:05.661 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:05.662 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:05.662 INFO:teuthology.orchestra.run.smithi032.stdout:2044 punch from 0xa65202 to 0xa7516f, (0xff6d bytes) 2024-09-13T09:18:05.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:05.664 INFO:teuthology.orchestra.run.smithi032.stdout:2046 write 0x6ab18e3 thru 0x6aba6e1 (0x8dff bytes) 2024-09-13T09:18:05.664 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:05.664 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:05.664 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:05.665 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:05.665 INFO:teuthology.orchestra.run.smithi032.stdout:2047 writesame 0xea2a3e0 thru 0xea33a37 data_size 0x44(0x9658 bytes) 2024-09-13T09:18:05.665 INFO:teuthology.orchestra.run.smithi032.stdout:2048 read 0x8f173a1 thru 0x8f23f1c (0xcb7c bytes) 2024-09-13T09:18:05.665 INFO:teuthology.orchestra.run.smithi032.stdout:2049 trunc from 0xea33a38 to 0x7d25cc2 2024-09-13T09:18:05.666 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:05.666 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:05.667 INFO:teuthology.orchestra.run.smithi032.stdout:2050 read 0x64c77e0 thru 0x64c934d (0x1b6e bytes) 2024-09-13T09:18:05.667 INFO:teuthology.orchestra.run.smithi032.stdout:2052 trunc from 0x7d25cc2 to 0xfdf707 2024-09-13T09:18:05.687 INFO:teuthology.orchestra.run.smithi032.stdout:2053 write 0xb036f83 thru 0xb04286d (0xb8eb bytes) 2024-09-13T09:18:05.687 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:05.684+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:05.693 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:05.707 INFO:teuthology.orchestra.run.smithi032.stdout:2055 write 0xe6de66d thru 0xe6e0a48 (0x23dc bytes) 2024-09-13T09:18:05.716 INFO:teuthology.orchestra.run.smithi032.stdout:2056 trunc from 0xe6e0a49 to 0xd15b680 2024-09-13T09:18:05.738 INFO:teuthology.orchestra.run.smithi032.stdout:2057 read 0x7afe64a thru 0x7b0ab28 (0xc4df bytes) 2024-09-13T09:18:05.738 INFO:teuthology.orchestra.run.smithi032.stdout:2058 write 0x5e7aaf1 thru 0x5e8a316 (0xf826 bytes) 2024-09-13T09:18:05.749 INFO:teuthology.orchestra.run.smithi032.stdout:2059 write 0xbbd4946 thru 0xbbdefcc (0xa687 bytes) 2024-09-13T09:18:05.758 INFO:teuthology.orchestra.run.smithi032.stdout:2062 read 0x90754d6 thru 0x9078c27 (0x3752 bytes) 2024-09-13T09:18:05.758 INFO:teuthology.orchestra.run.smithi032.stdout:2063 read 0x94962a1 thru 0x9497f52 (0x1cb2 bytes) 2024-09-13T09:18:05.758 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:05.758 INFO:teuthology.orchestra.run.smithi032.stdout:2065 write 0x19c0ce6 thru 0x19c2079 (0x1394 bytes) 2024-09-13T09:18:05.764 INFO:teuthology.orchestra.run.smithi032.stdout:2066 writesame 0xc8a98e5 thru 0xc8aa4e0 data_size 0x4(0xbfc bytes) 2024-09-13T09:18:05.765 INFO:teuthology.orchestra.run.smithi032.stdout:2067 write 0xac247f2 thru 0xac31577 (0xcd86 bytes) 2024-09-13T09:18:05.768 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:05.764+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:05.773 INFO:teuthology.orchestra.run.smithi032.stdout:2068 punch from 0x5800aa5 to 0x5800d98, (0x2f3 bytes) 2024-09-13T09:18:05.773 INFO:teuthology.orchestra.run.smithi032.stdout:2070 read 0x1a41b1c thru 0x1a44678 (0x2b5d bytes) 2024-09-13T09:18:05.774 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:05.784 INFO:teuthology.orchestra.run.smithi032.stdout:2073 trunc from 0xd15b680 to 0x8e3a663 2024-09-13T09:18:05.807 INFO:teuthology.orchestra.run.smithi032.stdout:2074 write 0x68ab681 thru 0x68ad1ad (0x1b2d bytes) 2024-09-13T09:18:05.810 INFO:teuthology.orchestra.run.smithi032.stdout:2075 punch from 0x4875273 to 0x4880fdd, (0xbd6a bytes) 2024-09-13T09:18:05.813 INFO:teuthology.orchestra.run.smithi032.stdout:2076 trunc from 0x8e3a663 to 0x9d02674 2024-09-13T09:18:05.821 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:05.821 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:05.821 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:05.821 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:05.833 INFO:teuthology.orchestra.run.smithi032.stdout:2077 write 0xa3a70ac thru 0xa3a8c17 (0x1b6c bytes) 2024-09-13T09:18:05.837 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:05.840 INFO:teuthology.orchestra.run.smithi032.stdout:2078 punch from 0x1cf8213 to 0x1cfa41d, (0x220a bytes) 2024-09-13T09:18:05.841 INFO:teuthology.orchestra.run.smithi032.stdout:2080 write 0x5d4ee37 thru 0x5d502c2 (0x148c bytes) 2024-09-13T09:18:05.848 INFO:teuthology.orchestra.run.smithi032.stdout:2081 writesame 0x8585bc9 thru 0x858e753 data_size 0x1(0x8b8b bytes) 2024-09-13T09:18:05.869 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:05.864+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:05.902 INFO:teuthology.orchestra.run.smithi032.stdout:2082 read 0x7d4b3f0 thru 0x7d59e20 (0xea31 bytes) 2024-09-13T09:18:05.904 INFO:teuthology.orchestra.run.smithi032.stdout:2083 write 0x31a8bca thru 0x31b77dd (0xec14 bytes) 2024-09-13T09:18:05.906 INFO:teuthology.orchestra.run.smithi032.stdout:2085 read 0x55a1fd2 thru 0x55ac900 (0xa92f bytes) 2024-09-13T09:18:05.921 INFO:teuthology.orchestra.run.smithi032.stdout:2087 write 0xd7fba5f thru 0xd808343 (0xc8e5 bytes) 2024-09-13T09:18:05.930 INFO:teuthology.orchestra.run.smithi032.stdout:2088 write 0xbdc8edb thru 0xbdd8292 (0xf3b8 bytes) 2024-09-13T09:18:05.938 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:05.938 INFO:teuthology.orchestra.run.smithi032.stdout:2090 writesame 0xb29ace thru 0xb37240 data_size 0x5(0xd773 bytes) 2024-09-13T09:18:05.947 INFO:teuthology.orchestra.run.smithi032.stdout:2092 punch from 0x22e7606 to 0x22f0b1b, (0x9515 bytes) 2024-09-13T09:18:05.951 INFO:teuthology.orchestra.run.smithi032.stdout:2093 trunc from 0xd808344 to 0xb2205bb 2024-09-13T09:18:05.969 INFO:teuthology.orchestra.run.smithi032.stdout:2094 punch from 0x59ae452 to 0x59bb7a7, (0xd355 bytes) 2024-09-13T09:18:05.970 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:05.968+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:05.970 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:05.970 INFO:teuthology.orchestra.run.smithi032.stdout:2096 punch from 0x1043f4 to 0x105c89, (0x1895 bytes) 2024-09-13T09:18:05.971 INFO:teuthology.orchestra.run.smithi032.stdout:2097 read 0x909393b thru 0x909dc16 (0xa2dc bytes) 2024-09-13T09:18:05.971 INFO:teuthology.orchestra.run.smithi032.stdout:2099 punch from 0xa1efc58 to 0xa1efdb1, (0x159 bytes) 2024-09-13T09:18:05.971 INFO:teuthology.orchestra.run.smithi032.stdout:2100 writesame 0x3e18ffc thru 0x3e21685 data_size 0x11(0x868a bytes) 2024-09-13T09:18:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-13T09:18:05.977 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:05.977 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:05.977 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:05.977 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:05.977 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:05.997 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:05.999 INFO:teuthology.orchestra.run.smithi032.stdout:2101 write 0xcafdcb0 thru 0xcb0d834 (0xfb85 bytes) 2024-09-13T09:18:06.008 INFO:teuthology.orchestra.run.smithi032.stdout:2103 write 0x236c5bb thru 0x237c32f (0xfd75 bytes) 2024-09-13T09:18:06.010 INFO:teuthology.orchestra.run.smithi032.stdout:2104 writesame 0x3881670 thru 0x3883843 data_size 0x1(0x21d4 bytes) 2024-09-13T09:18:06.027 INFO:teuthology.orchestra.run.smithi032.stdout:2106 punch from 0x6bbfcb5 to 0x6bc4f80, (0x52cb bytes) 2024-09-13T09:18:06.027 INFO:teuthology.orchestra.run.smithi032.stdout:2108 writesame 0x61318ff thru 0x613db98 data_size 0x72(0xc29a bytes) 2024-09-13T09:18:06.030 INFO:teuthology.orchestra.run.smithi032.stdout:2109 write 0x8d26f45 thru 0x8d287fa (0x18b6 bytes) 2024-09-13T09:18:06.037 INFO:teuthology.orchestra.run.smithi032.stdout:2113 read 0x2656204 thru 0x265953a (0x3337 bytes) 2024-09-13T09:18:06.040 INFO:teuthology.orchestra.run.smithi032.stdout:2114 trunc from 0xcb0d835 to 0xae50580 2024-09-13T09:18:06.052 INFO:teuthology.orchestra.run.smithi032.stdout:2115 read 0x824904e thru 0x8249d70 (0xd23 bytes) 2024-09-13T09:18:06.053 INFO:teuthology.orchestra.run.smithi032.stdout:2116 writesame 0x81826eb thru 0x8183099 data_size 0x1(0x9af bytes) 2024-09-13T09:18:06.067 INFO:teuthology.orchestra.run.smithi032.stdout:2117 write 0xb0fd3f0 thru 0xb0ff37a (0x1f8b bytes) 2024-09-13T09:18:06.071 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:06.066+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:06.135 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:06.135 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:06.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:06.135 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:06.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.136 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5478 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.136 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:18:06.136 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.594733115, 2024-09-13T09:18:06.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027749532, 2024-09-13T09:18:06.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.136 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.136 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.136 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.136 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5478 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539234+0000", 2024-09-13T09:18:06.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539236+0000", 2024-09-13T09:18:06.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.041e-06 2024-09-13T09:18:06.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539242+0000", 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0650000000000004e-06 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539246+0000", 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1500000000000001e-06 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566770+0000", 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027523178999999998 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566846+0000", 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5941999999999999e-05 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.566984+0000", 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000138486 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.140 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5479 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.594716851000001, 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027937460000000001, 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5479 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539250+0000", 2024-09-13T09:18:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539256+0000", 2024-09-13T09:18:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:18:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539263+0000", 2024-09-13T09:18:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3779999999999998e-06 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539273+0000", 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0457e-05 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567010+0000", 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027736878 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567083+0000", 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3187000000000002e-05 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567188+0000", 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010491499999999999 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.145 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5480 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.594689633000002, 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028132641999999999, 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5480 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539280+0000", 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539297+0000", 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7147999999999999e-05 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539303+0000", 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9039999999999997e-06 2024-09-13T09:18:06.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567230+0000", 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027927250000000001 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567316+0000", 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5970999999999994e-05 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567411+0000", 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4523000000000003e-05 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5481 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.594659636999999, 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028317978000000001, 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.150 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5481 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:18:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539307+0000", 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539311+0000", 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.264e-06 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539317+0000", 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539322+0000", 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6949999999999999e-06 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567446+0000", 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028124341000000001 2024-09-13T09:18:06.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567527+0000", 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0785000000000007e-05 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567626+0000", 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9103000000000003e-05 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.155 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5482 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.155 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:18:06.155 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.594641811999999, 2024-09-13T09:18:06.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028619326, 2024-09-13T09:18:06.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.155 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.155 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.155 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.155 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5482 2024-09-13T09:18:06.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.156 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:18:06.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539325+0000", 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539333+0000", 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.055e-06 2024-09-13T09:18:06.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539340+0000", 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4980000000000003e-06 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539344+0000", 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5369999999999999e-06 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567655+0000", 2024-09-13T09:18:06.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028310906 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567732+0000", 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6829999999999995e-05 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567945+0000", 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000212909 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5483 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.594618551, 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028784015999999999, 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5483 2024-09-13T09:18:06.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539351+0000", 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539357+0000", 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0090000000000004e-06 2024-09-13T09:18:06.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539361+0000", 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.357e-06 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567974+0000", 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028612385000000001 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568050+0000", 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5820000000000003e-05 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568133+0000", 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3551e-05 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5484 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.164 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.594601998999998, 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028949923999999998, 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5484 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:18:06.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:18:06.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539365+0000", 2024-09-13T09:18:06.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539367+0000", 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539374+0000", 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539378+0000", 2024-09-13T09:18:06.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568159+0000", 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028780490999999998 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568236+0000", 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7156000000000005e-05 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568316+0000", 2024-09-13T09:18:06.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9598999999999995e-05 2024-09-13T09:18:06.169 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.169 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.169 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.169 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5485 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.169 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:06.169 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.59378212, 2024-09-13T09:18:06.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028332593, 2024-09-13T09:18:06.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.169 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5485 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540189+0000", 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540198+0000", 2024-09-13T09:18:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9099999999999994e-06 2024-09-13T09:18:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540206+0000", 2024-09-13T09:18:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-13T09:18:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568360+0000", 2024-09-13T09:18:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028154324000000001 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568436+0000", 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5821000000000005e-05 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568518+0000", 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1895000000000001e-05 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5486 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.593729111000002, 2024-09-13T09:18:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028479536999999999, 2024-09-13T09:18:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5486 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540241+0000", 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4290000000000001e-06 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540249+0000", 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9480000000000008e-06 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540255+0000", 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6019999999999999e-06 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568543+0000", 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028288260999999999 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568636+0000", 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2627999999999995e-05 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568718+0000", 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2213999999999996e-05 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5487 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.593708026000002, 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028653435000000001, 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5487 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540259+0000", 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540262+0000", 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.413e-06 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540268+0000", 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3940000000000001e-06 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540272+0000", 2024-09-13T09:18:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3989999999999997e-06 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568745+0000", 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028472173999999999 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568822+0000", 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7694000000000005e-05 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568913+0000", 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0779000000000006e-05 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5488 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 430080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.593690626000001, 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028788326999999999, 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5488 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540279+0000", 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2409999999999998e-06 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540285+0000", 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3180000000000003e-06 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540300+0000", 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4625e-05 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568939+0000", 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028639688 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569016+0000", 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6556000000000004e-05 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569065+0000", 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9246000000000003e-05 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5489 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.593663031999998, 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028875332, 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5489 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540304+0000", 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540307+0000", 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4370000000000001e-06 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540313+0000", 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540318+0000", 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:18:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569079+0000", 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028760714 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569128+0000", 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8702999999999998e-05 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569180+0000", 2024-09-13T09:18:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2105999999999999e-05 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5490 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.593644536999999, 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029041027000000001, 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5490 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540326+0000", 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9840000000000001e-06 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540332+0000", 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9730000000000002e-06 2024-09-13T09:18:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540338+0000", 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3550000000000001e-06 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569202+0000", 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028863521 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569276+0000", 2024-09-13T09:18:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569364+0000", 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7594000000000002e-05 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5491 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.593625153000001, 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029221864, 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5491 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540350+0000", 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4250000000000001e-06 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540356+0000", 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2960000000000004e-06 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540361+0000", 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569388+0000", 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029026744 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569474+0000", 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6519000000000003e-05 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569564+0000", 2024-09-13T09:18:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9969999999999994e-05 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5492 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.593601855999999, 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029401203000000001, 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5492 2024-09-13T09:18:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:18:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:18:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540365+0000", 2024-09-13T09:18:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540368+0000", 2024-09-13T09:18:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540374+0000", 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9349999999999999e-06 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540378+0000", 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.651e-06 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569591+0000", 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029212557 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569668+0000", 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569767+0000", 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9061000000000006e-05 2024-09-13T09:18:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5493 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.593584992, 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029599424999999999, 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5493 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540382+0000", 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540384+0000", 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0430000000000002e-06 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540390+0000", 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540395+0000", 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5990000000000004e-06 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569795+0000", 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029400582000000001 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569880+0000", 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5210999999999994e-05 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569982+0000", 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101584 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5494 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.593568907000002, 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029770637999999999, 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5494 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:18:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540398+0000", 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540406+0000", 2024-09-13T09:18:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540413+0000", 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540418+0000", 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8289999999999998e-06 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570007+0000", 2024-09-13T09:18:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029588851999999999 2024-09-13T09:18:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570084+0000", 2024-09-13T09:18:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7253e-05 2024-09-13T09:18:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570169+0000", 2024-09-13T09:18:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5391999999999998e-05 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5495 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.593530591, 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029938664, 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5495 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540440+0000", 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0319999999999999e-06 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540446+0000", 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8549999999999996e-06 2024-09-13T09:18:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540451+0000", 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570198+0000", 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029746060000000001 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570284+0000", 2024-09-13T09:18:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6044999999999994e-05 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570376+0000", 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2071000000000007e-05 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5496 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.593511635999999, 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030121617999999999, 2024-09-13T09:18:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5496 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540458+0000", 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540464+0000", 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.156e-06 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540468+0000", 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570404+0000", 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029935632 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570488+0000", 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3873000000000001e-05 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570578+0000", 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9757000000000001e-05 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5497 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:18:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.593495072, 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030305259000000001, 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5497 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:18:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540472+0000", 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540474+0000", 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0490000000000002e-06 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540480+0000", 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1970000000000001e-06 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540491+0000", 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0964e-05 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570608+0000", 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030116105000000001 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570688+0000", 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0143999999999997e-05 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570778+0000", 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0131999999999997e-05 2024-09-13T09:18:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.228 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:06.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:06.230 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:06.224+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:06.330 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:06.328+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:06.405 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:06.405 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:06.405 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:06.405 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:06.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:06.462 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:06.460+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:06.532 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:06.528+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:06.590 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:06.590 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:06.590 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:06.591 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:06.591 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:06.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:06.633 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:06.626+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:06.734 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:06.730+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:06.778 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3022 3.9 e31/22 3:91681905:::rbd_data.118591b29fad.0000000000000209:head v 31'171, mlcod=31'169)", 2024-09-13T09:18:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:18:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.307810042, 2024-09-13T09:18:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002956209, 2024-09-13T09:18:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468240+0000", 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468237+0000", 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468300+0000", 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3202999999999998e-05 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468353+0000", 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2735999999999998e-05 2024-09-13T09:18:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468374+0000", 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0930000000000001e-05 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468568+0000", 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000194262 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.468627+0000", 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8743000000000003e-05 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.471132+0000", 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025055540000000001 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:38.471196+0000", 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3590999999999995e-05 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "pg_update_log_missing(3.4 epoch 35/22 rep_tid 113 entries 35'31 (0'0) error 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head by client.4485.0:3311 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 31'29)", 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.248129529, 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033204409999999999, 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:18:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527918+0000", 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527943+0000", 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4752999999999998e-05 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527988+0000", 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4693000000000001e-05 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528029+0000", 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1155999999999997e-05 2024-09-13T09:18:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528194+0000", 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164792 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.531241+0000", 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003046867 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3323 3.4 e35/22 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head v 35'34, mlcod=35'31)", 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.240254603, 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037621640000000001, 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535793+0000", 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535811+0000", 2024-09-13T09:18:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8170000000000001e-05 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535840+0000", 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9105999999999999e-05 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535877+0000", 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7017000000000002e-05 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536001+0000", 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012342899999999999 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536073+0000", 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2123e-05 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539433+0000", 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033596630000000001 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539557+0000", 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012470600000000001 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:15977 3.4 3:27c24f46:::rbd_data.11856af9a4d3.0000000000000005:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.78098509, 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003633163, 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15977 2024-09-13T09:18:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:18:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:18:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995063+0000", 2024-09-13T09:18:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995071+0000", 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3930000000000002e-06 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995086+0000", 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4970999999999999e-05 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995095+0000", 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8810000000000002e-06 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995180+0000", 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5012000000000005e-05 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995378+0000", 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019788600000000001 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.998698+0000", 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003320267 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16010 3.4 3:27c24f46:::rbd_data.11856af9a4d3.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.775376297999999, 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037539579999999999, 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16010 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000672+0000", 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000679+0000", 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9120000000000001e-06 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000693+0000", 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3745e-05 2024-09-13T09:18:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000702+0000", 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.064e-06 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000763+0000", 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1309000000000005e-05 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000877+0000", 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114214 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.001145+0000", 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000267765 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.002514+0000", 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013688730000000001 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004301+0000", 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017869839999999999 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004380+0000", 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8861000000000003e-05 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004427+0000", 2024-09-13T09:18:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7713000000000002e-05 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16047 3.0 3:002d813e:::rbd_data.11856af9a4d3.0000000000000014:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.766499295999999, 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030667929999999999, 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16047 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009548+0000", 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009559+0000", 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0936e-05 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009579+0000", 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9976e-05 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009590+0000", 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1228e-05 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009670+0000", 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9765999999999995e-05 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009776+0000", 2024-09-13T09:18:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106049 2024-09-13T09:18:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:18:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009991+0000", 2024-09-13T09:18:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021494999999999999 2024-09-13T09:18:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.011621+0000", 2024-09-13T09:18:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016294650000000001 2024-09-13T09:18:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012469+0000", 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000848598 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012551+0000", 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2144999999999994e-05 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012617+0000", 2024-09-13T09:18:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5827999999999994e-05 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16226 3.4 3:22419c10:::rbd_data.1185f20bdb42.0000000000000013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.547499192, 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033492510000000001, 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16226 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228546+0000", 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999957 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228569+0000", 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2347000000000001e-05 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228618+0000", 2024-09-13T09:18:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9509999999999999e-05 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228643+0000", 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4759000000000001e-05 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228738+0000", 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5273999999999996e-05 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228819+0000", 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0623000000000004e-05 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228921+0000", 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010215199999999999 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.230000+0000", 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010793510000000001 2024-09-13T09:18:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231718+0000", 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001717873 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231813+0000", 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4473000000000002e-05 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231900+0000", 2024-09-13T09:18:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7158999999999997e-05 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16293 3.4 3:22419c10:::rbd_data.1185f20bdb42.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.393888511, 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015655326000000001, 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16293 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382158+0000", 2024-09-13T09:18:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382176+0000", 2024-09-13T09:18:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7863000000000002e-05 2024-09-13T09:18:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382211+0000", 2024-09-13T09:18:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.481e-05 2024-09-13T09:18:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382226+0000", 2024-09-13T09:18:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.537e-05 2024-09-13T09:18:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382354+0000", 2024-09-13T09:18:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012803599999999999 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382591+0000", 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023629099999999999 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.385269+0000", 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002678532 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397515+0000", 2024-09-13T09:18:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012245946000000001 2024-09-13T09:18:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397551+0000", 2024-09-13T09:18:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5856999999999999e-05 2024-09-13T09:18:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397699+0000", 2024-09-13T09:18:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148255 2024-09-13T09:18:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397817+0000", 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117299 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4485.0:16293 3.4 e42/41 ondisk, result = 0)", 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.383556971000001, 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0050343319999999999, 2024-09-13T09:18:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:18:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:18:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392491+0000", 2024-09-13T09:18:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392500+0000", 2024-09-13T09:18:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8300000000000002e-06 2024-09-13T09:18:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392529+0000", 2024-09-13T09:18:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9260000000000001e-05 2024-09-13T09:18:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392542+0000", 2024-09-13T09:18:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2969999999999999e-05 2024-09-13T09:18:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397468+0000", 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0049256880000000001 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397496+0000", 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8209999999999999e-05 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397527+0000", 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1276999999999999e-05 2024-09-13T09:18:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16837 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [call rbd.sparse_copyup in=173207b,set-alloc-hint object_size 16777216 write_size 16777216,write 3145728~10909 in=10909b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:18:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:18:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.515926157999999, 2024-09-13T09:18:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004073887, 2024-09-13T09:18:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16837 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260121+0000", 2024-09-13T09:18:06.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:06.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260266+0000", 2024-09-13T09:18:06.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014495999999999999 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260287+0000", 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1336e-05 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260296+0000", 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6170000000000007e-06 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260352+0000", 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6252999999999998e-05 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260520+0000", 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000168381 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260807+0000", 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028624100000000001 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.262062+0000", 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012551490000000001 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264128+0000", 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020660380000000001 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264167+0000", 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9714e-05 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264198+0000", 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0105000000000001e-05 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:17005 3.1a e43/41 3:5e94b56b:::rbd_data.11859d3d11dd.0000000000000008:head v 43'54, mlcod=42'52)", 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.390361500999999, 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031754880000000002, 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:06.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.386997+0000", 2024-09-13T09:18:06.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00130933 2024-09-13T09:18:06.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387013+0000", 2024-09-13T09:18:06.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5752000000000001e-05 2024-09-13T09:18:06.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387022+0000", 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1689999999999998e-06 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387092+0000", 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9385000000000001e-05 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387127+0000", 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5061000000000002e-05 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.388821+0000", 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001694463 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.388864+0000", 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2721e-05 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:17028 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [stat,zero 4194304~4718592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:18:06.835 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.366543048, 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036807770000000001, 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 17028 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409504+0000", 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409515+0000", 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0485e-05 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409540+0000", 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4808e-05 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409550+0000", 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0376e-05 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409616+0000", 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6130999999999994e-05 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409725+0000", 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108614 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.413188+0000", 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003462849 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.839 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18757 3.4 3:21c7a0ac:::rbd_data.1185b72576e4.0000000000000074:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.146015355999999, 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066789190000000002, 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18757 2024-09-13T09:18:06.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630032+0000", 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630044+0000", 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2245999999999999e-05 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630067+0000", 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3404000000000002e-05 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630078+0000", 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1158999999999999e-05 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630156+0000", 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7520000000000003e-05 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630357+0000", 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020064700000000001 2024-09-13T09:18:06.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.631878+0000", 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015212940000000001 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636574+0000", 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046960719999999999 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:06.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636612+0000", 2024-09-13T09:18:06.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7499999999999997e-05 2024-09-13T09:18:06.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636669+0000", 2024-09-13T09:18:06.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7286e-05 2024-09-13T09:18:06.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636713+0000", 2024-09-13T09:18:06.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4530000000000002e-05 2024-09-13T09:18:06.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18821 3.4 3:250382e8:::rbd_data.1185c2de398f.0000000000000005:e [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.648454+0000", 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.127595712, 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002981521, 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18821 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648454+0000", 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648454+0000", 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648454+0000", 2024-09-13T09:18:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-13T09:18:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648455+0000", 2024-09-13T09:18:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.066e-06 2024-09-13T09:18:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648458+0000", 2024-09-13T09:18:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3340000000000002e-06 2024-09-13T09:18:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648461+0000", 2024-09-13T09:18:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-13T09:18:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651241+0000", 2024-09-13T09:18:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002779625 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651336+0000", 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4946999999999997e-05 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651436+0000", 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9925000000000003e-05 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18822 3.4 3:250382e8:::rbd_data.1185c2de398f.0000000000000005:e [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:18:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.127586271, 2024-09-13T09:18:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031435460000000001, 2024-09-13T09:18:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18822 2024-09-13T09:18:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648463+0000", 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-06 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648468+0000", 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1999999999999999e-06 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648470+0000", 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7389999999999999e-06 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651484+0000", 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003014128 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651541+0000", 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6304999999999997e-05 2024-09-13T09:18:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651607+0000", 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6280999999999998e-05 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19013 3.4 3:21c7a0ac:::rbd_data.1185b72576e4.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.874194+0000", 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.9018558120000009, 2024-09-13T09:18:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033037850000000001, 2024-09-13T09:18:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19013 2024-09-13T09:18:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874194+0000", 2024-09-13T09:18:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874194+0000", 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874191+0000", 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874207+0000", 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6441000000000001e-05 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874235+0000", 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7554000000000001e-05 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874246+0000", 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1114999999999999e-05 2024-09-13T09:18:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874331+0000", 2024-09-13T09:18:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4608e-05 2024-09-13T09:18:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874450+0000", 2024-09-13T09:18:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011943900000000001 2024-09-13T09:18:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874607+0000", 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015705600000000001 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.875492+0000", 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00088451199999999995 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.877353+0000", 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018616220000000001 2024-09-13T09:18:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.877437+0000", 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3698999999999999e-05 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.877498+0000", 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0745999999999997e-05 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19037 3.4 3:277a4fb8:::rbd_data.1185b72576e4.000000000000015a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.883824+0000", 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.8922255030000006, 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029514070000000001, 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19037 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883824+0000", 2024-09-13T09:18:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883824+0000", 2024-09-13T09:18:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883822+0000", 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883836+0000", 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3910000000000001e-05 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883891+0000", 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4639999999999999e-05 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.883912+0000", 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1043000000000002e-05 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.884074+0000", 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000162039 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.884308+0000", 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023380800000000001 2024-09-13T09:18:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.884623+0000", 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031597400000000001 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.886279+0000", 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001655091 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.886674+0000", 2024-09-13T09:18:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039576299999999999 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.886733+0000", 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8390000000000002e-05 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.886776+0000", 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2969000000000001e-05 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:20630 3.4 3:23dc35fd:::rbd_data.1185b72576e4.000000000000007c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:00.986895+0000", 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7891549329999998, 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0040477539999999998, 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 20630 2024-09-13T09:18:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986895+0000", 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986895+0000", 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986893+0000", 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986899+0000", 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986910+0000", 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1105e-05 2024-09-13T09:18:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986917+0000", 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986963+0000", 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5932e-05 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.987020+0000", 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6858999999999999e-05 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.987104+0000", 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4250000000000001e-05 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.987786+0000", 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00068249499999999998 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.990828+0000", 2024-09-13T09:18:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030412099999999999 2024-09-13T09:18:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.990893+0000", 2024-09-13T09:18:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5117999999999996e-05 2024-09-13T09:18:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.990943+0000", 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9789000000000001e-05 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:21345 3.0 3:004c516c:::rbd_header.1185d6f8ec87:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:02.094060+0000", 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6819899569999999, 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004611782, 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 21345 2024-09-13T09:18:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094060+0000", 2024-09-13T09:18:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094060+0000", 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094057+0000", 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094070+0000", 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2787e-05 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094095+0000", 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4873000000000001e-05 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094106+0000", 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0954e-05 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094171+0000", 2024-09-13T09:18:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4958999999999999e-05 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094344+0000", 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000173098 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094590+0000", 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024549199999999999 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.095579+0000", 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00098965800000000003 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.098585+0000", 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030058390000000002 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.098630+0000", 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5393999999999999e-05 2024-09-13T09:18:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.098672+0000", 2024-09-13T09:18:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1153999999999999e-05 2024-09-13T09:18:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:21855 3.0 3:004c516c:::rbd_header.1185d6f8ec87:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:18:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:02.581557+0000", 2024-09-13T09:18:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.194492726, 2024-09-13T09:18:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028030970000000001, 2024-09-13T09:18:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 21855 2024-09-13T09:18:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.581557+0000", 2024-09-13T09:18:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.581557+0000", 2024-09-13T09:18:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.581554+0000", 2024-09-13T09:18:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.581571+0000", 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6548000000000001e-05 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.581606+0000", 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5287000000000003e-05 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.581624+0000", 2024-09-13T09:18:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8482e-05 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.581727+0000", 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102934 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.581883+0000", 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015576199999999999 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.582301+0000", 2024-09-13T09:18:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000417971 2024-09-13T09:18:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.583530+0000", 2024-09-13T09:18:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00122938 2024-09-13T09:18:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.584231+0000", 2024-09-13T09:18:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00070007399999999997 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.584305+0000", 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4209999999999996e-05 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.584360+0000", 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5389000000000001e-05 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:06.887 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:06.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:06.889 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:06.884+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:06.976 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:06.976 INFO:teuthology.orchestra.run.smithi032.stdout:2119 read 0x5ed4efc thru 0x5ee2fd5 (0xe0da bytes) 2024-09-13T09:18:06.989 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:06.984+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:06.998 INFO:teuthology.orchestra.run.smithi032.stdout:2120 trunc from 0xb0ff37b to 0x21d6dd0 2024-09-13T09:18:07.031 INFO:teuthology.orchestra.run.smithi032.stdout:2121 punch from 0x1d030f1 to 0x1d0861e, (0x552d bytes) 2024-09-13T09:18:07.031 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.034 INFO:teuthology.orchestra.run.smithi032.stdout:2123 trunc from 0x21d6dd0 to 0x6de08b 2024-09-13T09:18:07.049 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.080 INFO:teuthology.orchestra.run.smithi032.stdout:2125 write 0x89c232c thru 0x89c6ed4 (0x4ba9 bytes) 2024-09-13T09:18:07.084 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:07.085 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:07.085 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:07.085 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:07.088 INFO:teuthology.orchestra.run.smithi032.stdout:2126 read 0x76f2371 thru 0x76fad1f (0x89af bytes) 2024-09-13T09:18:07.089 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:07.084+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:07.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:07.109 INFO:teuthology.orchestra.run.smithi032.stdout:2127 write 0xdd09419 thru 0xdd155b2 (0xc19a bytes) 2024-09-13T09:18:07.117 INFO:teuthology.orchestra.run.smithi032.stdout:2128 read 0xc96877 thru 0xca1f32 (0xb6bc bytes) 2024-09-13T09:18:07.118 INFO:teuthology.orchestra.run.smithi032.stdout:2129 read 0x924af68 thru 0x92573da (0xc473 bytes) 2024-09-13T09:18:07.118 INFO:teuthology.orchestra.run.smithi032.stdout:2130 read 0x9b014f8 thru 0x9b0c690 (0xb199 bytes) 2024-09-13T09:18:07.118 INFO:teuthology.orchestra.run.smithi032.stdout:2131 read 0x329961f thru 0x32a34e2 (0x9ec4 bytes) 2024-09-13T09:18:07.119 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.128 INFO:teuthology.orchestra.run.smithi032.stdout:2133 trunc from 0xdd155b3 to 0x865f98d 2024-09-13T09:18:07.151 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.151 INFO:teuthology.orchestra.run.smithi032.stdout:2135 punch from 0x64a057a to 0x64a39c0, (0x3446 bytes) 2024-09-13T09:18:07.159 INFO:teuthology.orchestra.run.smithi032.stdout:2136 trunc from 0x865f98d to 0xd2f59d6 2024-09-13T09:18:07.174 INFO:teuthology.orchestra.run.smithi032.stdout:2137 trunc from 0xd2f59d6 to 0x9d2cb1b 2024-09-13T09:18:07.188 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.188 INFO:teuthology.orchestra.run.smithi032.stdout:2139 write 0x236e2c2 thru 0x2377097 (0x8dd6 bytes) 2024-09-13T09:18:07.189 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:07.184+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:07.192 INFO:teuthology.orchestra.run.smithi032.stdout:2141 punch from 0x177386 to 0x185686, (0xe300 bytes) 2024-09-13T09:18:07.192 INFO:teuthology.orchestra.run.smithi032.stdout:2143 read 0x504ddec thru 0x5053c73 (0x5e88 bytes) 2024-09-13T09:18:07.192 INFO:teuthology.orchestra.run.smithi032.stdout:2144 write 0x94baff9 thru 0x94c6d8b (0xbd93 bytes) 2024-09-13T09:18:07.198 INFO:teuthology.orchestra.run.smithi032.stdout:2145 read 0x64476ce thru 0x644ec4e (0x7581 bytes) 2024-09-13T09:18:07.199 INFO:teuthology.orchestra.run.smithi032.stdout:2147 write 0x3c3ace9 thru 0x3c44cd2 (0x9fea bytes) 2024-09-13T09:18:07.206 INFO:teuthology.orchestra.run.smithi032.stdout:2148 read 0x40c685f thru 0x40d4b74 (0xe316 bytes) 2024-09-13T09:18:07.206 INFO:teuthology.orchestra.run.smithi032.stdout:2149 write 0x1bb8a51 thru 0x1bc79b9 (0xef69 bytes) 2024-09-13T09:18:07.214 INFO:teuthology.orchestra.run.smithi032.stdout:2150 read 0x7a13bf8 thru 0x7a1a811 (0x6c1a bytes) 2024-09-13T09:18:07.214 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.214 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.214 INFO:teuthology.orchestra.run.smithi032.stdout:2153 write 0x18f050c thru 0x18fe6bd (0xe1b2 bytes) 2024-09-13T09:18:07.224 INFO:teuthology.orchestra.run.smithi032.stdout:2154 trunc from 0x9d2cb1b to 0x68ad255 2024-09-13T09:18:07.243 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:07.243 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:07.243 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:07.243 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:07.244 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:07.250 INFO:teuthology.orchestra.run.smithi032.stdout:2155 punch from 0x2b11852 to 0x2b1cb88, (0xb336 bytes) 2024-09-13T09:18:07.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:07.263 INFO:teuthology.orchestra.run.smithi032.stdout:2157 write 0x7493c80 thru 0x7497474 (0x37f5 bytes) 2024-09-13T09:18:07.269 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.283 INFO:teuthology.orchestra.run.smithi032.stdout:2159 trunc from 0x7497475 to 0xe753a1 2024-09-13T09:18:07.290 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:07.286+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:07.307 INFO:teuthology.orchestra.run.smithi032.stdout:2161 read 0x2b063c thru 0x2bc896 (0xc25b bytes) 2024-09-13T09:18:07.307 INFO:teuthology.orchestra.run.smithi032.stdout:2163 read 0x8cbf29 thru 0x8d6bb4 (0xac8c bytes) 2024-09-13T09:18:07.308 INFO:teuthology.orchestra.run.smithi032.stdout:2164 read 0xcdbae6 thru 0xce1b07 (0x6022 bytes) 2024-09-13T09:18:07.309 INFO:teuthology.orchestra.run.smithi032.stdout:2165 read 0x58a52e thru 0x595aef (0xb5c2 bytes) 2024-09-13T09:18:07.326 INFO:teuthology.orchestra.run.smithi032.stdout:2167 write 0x5e79dda thru 0x5e7a54f (0x776 bytes) 2024-09-13T09:18:07.333 INFO:teuthology.orchestra.run.smithi032.stdout:2168 read 0x5b3e12b thru 0x5b45fda (0x7eb0 bytes) 2024-09-13T09:18:07.350 INFO:teuthology.orchestra.run.smithi032.stdout:2169 write 0xb53f02f thru 0xb54dd94 (0xed66 bytes) 2024-09-13T09:18:07.362 INFO:teuthology.orchestra.run.smithi032.stdout:2171 trunc from 0xb54dd95 to 0xe7680f9 2024-09-13T09:18:07.372 INFO:teuthology.orchestra.run.smithi032.stdout:2173 write 0xcf6ac34 thru 0xcf72deb (0x81b8 bytes) 2024-09-13T09:18:07.379 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.379 INFO:teuthology.orchestra.run.smithi032.stdout:2175 read 0x866935e thru 0x86711b7 (0x7e5a bytes) 2024-09-13T09:18:07.383 INFO:teuthology.orchestra.run.smithi032.stdout:2176 trunc from 0xe7680f9 to 0xbfcf637 2024-09-13T09:18:07.391 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:07.386+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:07.401 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:07.401 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.865332809999998, 2024-09-13T09:18:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:18:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:18:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:18:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19325 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2514944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.471856144, 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041428072000000003, 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19325 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929839+0000", 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929845+0000", 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9510000000000003e-06 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929850+0000", 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971169+0000", 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041318743999999998 2024-09-13T09:18:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971214+0000", 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971265+0000", 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0963e-05 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19326 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2613248~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.471839208, 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041534459000000003, 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19326 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929856+0000", 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0470000000000001e-06 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929862+0000", 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3300000000000004e-06 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929867+0000", 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971293+0000", 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041425785999999999 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971337+0000", 2024-09-13T09:18:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4240999999999999e-05 2024-09-13T09:18:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971389+0000", 2024-09-13T09:18:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1483999999999997e-05 2024-09-13T09:18:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19327 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2711552~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.471822142000001, 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041642875000000003, 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19327 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929873+0000", 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.26e-06 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929879+0000", 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0519999999999998e-06 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929884+0000", 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:18:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971416+0000", 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041532122999999997 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971460+0000", 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4456999999999998e-05 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971514+0000", 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3579000000000003e-05 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19328 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2809856~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.471805378999999, 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041750662000000001, 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19328 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929890+0000", 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929896+0000", 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929901+0000", 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971542+0000", 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041640442 2024-09-13T09:18:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971587+0000", 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4972999999999998e-05 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971639+0000", 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1990000000000002e-05 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19329 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2908160~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.471763386999999, 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041833958999999997, 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19329 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929929+0000", 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929932+0000", 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0699999999999998e-06 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929941+0000", 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.149e-06 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929947+0000", 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.699e-06 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971666+0000", 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041719224999999999 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971711+0000", 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4823000000000001e-05 2024-09-13T09:18:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971764+0000", 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2741999999999997e-05 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19330 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 3006464~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.471740646000001, 2024-09-13T09:18:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041949980999999997, 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19330 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929952+0000", 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929955+0000", 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929961+0000", 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1199999999999999e-06 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929966+0000", 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971792+0000", 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041826680999999998 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971838+0000", 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5939000000000002e-05 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971903+0000", 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4499999999999996e-05 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19331 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 3104768~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:18:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.471723679, 2024-09-13T09:18:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042058848000000003, 2024-09-13T09:18:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19331 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929969+0000", 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929975+0000", 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.434e-06 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929981+0000", 2024-09-13T09:18:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929986+0000", 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1780000000000002e-06 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971931+0000", 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041944417999999997 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971976+0000", 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4962000000000001e-05 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972029+0000", 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2768e-05 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19332 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 65536~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.471696508999999, 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042158568, 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19332 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:18:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929996+0000", 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929999+0000", 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.306e-06 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930005+0000", 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0379999999999996e-06 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930010+0000", 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0969999999999996e-06 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972057+0000", 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042047117000000002 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972102+0000", 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5422e-05 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972155+0000", 2024-09-13T09:18:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2964999999999999e-05 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19333 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 163840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.471679419000001, 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042266805999999997, 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19333 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930016+0000", 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930022+0000", 2024-09-13T09:18:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2550000000000003e-06 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930027+0000", 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8060000000000002e-06 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972183+0000", 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042156276999999999 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972228+0000", 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5317e-05 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972281+0000", 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2398999999999998e-05 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19334 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 262144~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.471661574000001, 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042375573, 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19334 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930031+0000", 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930034+0000", 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930040+0000", 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0789999999999997e-06 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930045+0000", 2024-09-13T09:18:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0139999999999998e-06 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972307+0000", 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042262685000000001 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972352+0000", 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5021000000000001e-05 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972407+0000", 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4945000000000003e-05 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19335 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 360448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.47164476, 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042483785000000003, 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19335 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930048+0000", 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930050+0000", 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.136e-06 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930056+0000", 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930061+0000", 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9289999999999997e-06 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972435+0000", 2024-09-13T09:18:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042373347999999998 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972479+0000", 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4597e-05 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972532+0000", 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3034000000000001e-05 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19336 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 458752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.471627823, 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042590488000000003, 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19336 2024-09-13T09:18:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930065+0000", 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930067+0000", 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2110000000000001e-06 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930074+0000", 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4640000000000003e-06 2024-09-13T09:18:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930079+0000", 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7709999999999997e-06 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972560+0000", 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042481738999999998 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972604+0000", 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4060999999999997e-05 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972656+0000", 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.164e-05 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19337 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.471610648, 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042697997000000001, 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19337 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:18:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930082+0000", 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930085+0000", 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.221e-06 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930091+0000", 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.942e-06 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930096+0000", 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.04e-06 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972683+0000", 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042587885999999998 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972728+0000", 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4058999999999999e-05 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972781+0000", 2024-09-13T09:18:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3156999999999999e-05 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19342 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1048576~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.469693221, 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041418260999999998, 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19342 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932002+0000", 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3180000000000001e-06 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932008+0000", 2024-09-13T09:18:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2199999999999997e-06 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932013+0000", 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973319+0000", 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041305738000000002 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973364+0000", 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5809000000000001e-05 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973418+0000", 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3888000000000003e-05 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19343 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1146880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.469666014, 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041515107000000002, 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19343 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932034+0000", 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932040+0000", 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2260000000000002e-06 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932045+0000", 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5589999999999999e-06 2024-09-13T09:18:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973446+0000", 2024-09-13T09:18:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041401070999999998 2024-09-13T09:18:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973491+0000", 2024-09-13T09:18:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5021999999999997e-05 2024-09-13T09:18:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973542+0000", 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1465000000000003e-05 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19344 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.469643487999999, 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041617848999999998, 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19344 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932049+0000", 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932052+0000", 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4540000000000001e-06 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932064+0000", 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1911e-05 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932079+0000", 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4765e-05 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973570+0000", 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041491800000000002 2024-09-13T09:18:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973614+0000", 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4149e-05 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973668+0000", 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3198999999999997e-05 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19345 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1343488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.469607316999999, 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041707699000000001, 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19345 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932085+0000", 2024-09-13T09:18:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932089+0000", 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7009999999999998e-06 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932097+0000", 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3010000000000007e-06 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932104+0000", 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9260000000000003e-06 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973695+0000", 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041591056000000001 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973741+0000", 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6003e-05 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973794+0000", 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2265999999999997e-05 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19346 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1441792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.469583216, 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041808933999999999, 2024-09-13T09:18:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19346 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932112+0000", 2024-09-13T09:18:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932119+0000", 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8369999999999999e-06 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932125+0000", 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973822+0000", 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041697369999999997 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973867+0000", 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4648999999999999e-05 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973919+0000", 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2451000000000003e-05 2024-09-13T09:18:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19347 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1540096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.469563763, 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041915045999999997, 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19347 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932129+0000", 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932131+0000", 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.278e-06 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932139+0000", 2024-09-13T09:18:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.43e-06 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932145+0000", 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7939999999999999e-06 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973946+0000", 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041801724999999998 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973993+0000", 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6245000000000002e-05 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.974045+0000", 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1950999999999998e-05 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:07.495 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:07.496 INFO:teuthology.orchestra.run.smithi032.stdout:2178 trunc from 0xbfcf637 to 0x689e746 2024-09-13T09:18:07.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:07.498 INFO:teuthology.orchestra.run.smithi032.stdout:2179 write 0x4946d26 thru 0x4948623 (0x18fe bytes) 2024-09-13T09:18:07.498 INFO:teuthology.orchestra.run.smithi032.stdout:2183 trunc from 0x689e746 to 0x1b8ef08 2024-09-13T09:18:07.498 INFO:teuthology.orchestra.run.smithi032.stdout:2184 writesame 0x551ca64 thru 0x552450f data_size 0x2(0x7aac bytes) 2024-09-13T09:18:07.498 INFO:teuthology.orchestra.run.smithi032.stdout:2185 read 0x3b62fd6 thru 0x3b70cfe (0xdd29 bytes) 2024-09-13T09:18:07.498 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.499 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:07.490+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:07.522 INFO:teuthology.orchestra.run.smithi032.stdout:2187 write 0xbd5877f thru 0xbd670f0 (0xe972 bytes) 2024-09-13T09:18:07.529 INFO:teuthology.orchestra.run.smithi032.stdout:2188 read 0x3cefcd0 thru 0x3cf288b (0x2bbc bytes) 2024-09-13T09:18:07.529 INFO:teuthology.orchestra.run.smithi032.stdout:2189 trunc from 0xbd670f1 to 0xbaee461 2024-09-13T09:18:07.546 INFO:teuthology.orchestra.run.smithi032.stdout:2190 read 0x18f87e8 thru 0x19056c0 (0xced9 bytes) 2024-09-13T09:18:07.548 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.548 INFO:teuthology.orchestra.run.smithi032.stdout:2193 read 0x7c0d3ef thru 0x7c131ec (0x5dfe bytes) 2024-09-13T09:18:07.548 INFO:teuthology.orchestra.run.smithi032.stdout:2194 write 0x146d240 thru 0x14733ea (0x61ab bytes) 2024-09-13T09:18:07.551 INFO:teuthology.orchestra.run.smithi032.stdout:2195 punch from 0x1eb0371 to 0x1eb761e, (0x72ad bytes) 2024-09-13T09:18:07.551 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.551 INFO:teuthology.orchestra.run.smithi032.stdout:2198 write 0x8c14cfa thru 0x8c17433 (0x273a bytes) 2024-09-13T09:18:07.558 INFO:teuthology.orchestra.run.smithi032.stdout:2199 read 0x62b76e5 thru 0x62bd416 (0x5d32 bytes) 2024-09-13T09:18:07.578 INFO:teuthology.orchestra.run.smithi032.stdout:2200 trunc from 0xbaee461 to 0x49df67a 2024-09-13T09:18:07.598 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:07.594+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:07.602 INFO:teuthology.orchestra.run.smithi032.stdout:2201 punch from 0x4399d09 to 0x439c0ab, (0x23a2 bytes) 2024-09-13T09:18:07.602 INFO:teuthology.orchestra.run.smithi032.stdout:2202 read 0x3e45875 thru 0x3e48625 (0x2db1 bytes) 2024-09-13T09:18:07.602 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.602 INFO:teuthology.orchestra.run.smithi032.stdout:2205 read 0x2e8417c thru 0x2e8e8a2 (0xa727 bytes) 2024-09-13T09:18:07.603 INFO:teuthology.orchestra.run.smithi032.stdout:2206 writesame 0x1000d2e thru 0x100c765 data_size 0x8(0xba38 bytes) 2024-09-13T09:18:07.612 INFO:teuthology.orchestra.run.smithi032.stdout:2207 trunc from 0x49df67a to 0x4ed959c 2024-09-13T09:18:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:2209 read 0x142bae9 thru 0x14357ca (0x9ce2 bytes) 2024-09-13T09:18:07.628 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.637 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:07.637 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:07.638 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:07.638 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:07.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:07.658 INFO:teuthology.orchestra.run.smithi032.stdout:2214 writesame 0xa81d310 thru 0xa82362e data_size 0x1(0x631f bytes) 2024-09-13T09:18:07.668 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:18:07.686 INFO:teuthology.orchestra.run.smithi032.stdout:2215 trunc from 0xa82362f to 0xc14e4a 2024-09-13T09:18:07.741 INFO:teuthology.orchestra.run.smithi032.stdout:2216 write 0x643b2b thru 0x648dfc (0x52d2 bytes) 2024-09-13T09:18:07.741 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.741 INFO:teuthology.orchestra.run.smithi032.stdout:2218 read 0x80a62b thru 0x80e674 (0x404a bytes) 2024-09-13T09:18:07.742 INFO:teuthology.orchestra.run.smithi032.stdout:2219 trunc from 0xc14e4a to 0x5c981ae 2024-09-13T09:18:07.742 INFO:teuthology.orchestra.run.smithi032.stdout:2220 write 0x5c98982 thru 0x5ca7396 (0xea15 bytes) 2024-09-13T09:18:07.742 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.742 INFO:teuthology.orchestra.run.smithi032.stdout:2222 write 0x3a843c thru 0x3b804c (0xfc11 bytes) 2024-09-13T09:18:07.743 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:07.734+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:07.768 INFO:teuthology.orchestra.run.smithi032.stdout:2224 write 0xe1bfa3c thru 0xe1ca08c (0xa651 bytes) 2024-09-13T09:18:07.774 INFO:teuthology.orchestra.run.smithi032.stdout:2225 read 0x6346abb thru 0x6347784 (0xcca bytes) 2024-09-13T09:18:07.774 INFO:teuthology.orchestra.run.smithi032.stdout:2226 read 0xa132da1 thru 0xa1404aa (0xd70a bytes) 2024-09-13T09:18:07.775 INFO:teuthology.orchestra.run.smithi032.stdout:2228 punch from 0xb55fa8e to 0xb560e01, (0x1373 bytes) 2024-09-13T09:18:07.775 INFO:teuthology.orchestra.run.smithi032.stdout:2230 read 0xbc6f31 thru 0xbca6e2 (0x37b2 bytes) 2024-09-13T09:18:07.775 INFO:teuthology.orchestra.run.smithi032.stdout:2231 trunc from 0xe1ca08d to 0xdd689a7 2024-09-13T09:18:07.797 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.797 INFO:teuthology.orchestra.run.smithi032.stdout:2233 punch from 0xff9fe4 to 0xffef53, (0x4f6f bytes) 2024-09-13T09:18:07.797 INFO:teuthology.orchestra.run.smithi032.stdout:2234 punch from 0x616870a to 0x617342e, (0xad24 bytes) 2024-09-13T09:18:07.797 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:07.797 INFO:teuthology.orchestra.run.smithi032.stdout:2239 clone 13 order 25 su 16777216 sc 12 2024-09-13T09:18:07.830 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:07.842 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:07.838+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:07.846 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:07.846 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:07.862 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:07.862 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:07.872 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:07.873 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:07.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:07.878 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:07.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:07.898 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:07.898 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:07.916 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:07.916 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:07.936 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:07.943 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:07.939+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:07.955 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:07.956 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:07.984 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:18:07.984 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:18:07.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:18:08.044 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:08.038+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:08.062 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:08.062 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:08.064 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone12 from 0xdd689a7 (overlap 0x1804e9) to 0x159e04 2024-09-13T09:18:08.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:08.144 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:08.139+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:08.154 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-13T09:18:08.246 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:08.243+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:08.252 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:08.252 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:08.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:08.347 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:08.343+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:08.403 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:08.404 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:08.418 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:08.462 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:08.458+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:08.548 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:08.542+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:08.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:08.578 INFO:teuthology.orchestra.run.smithi032.stdout:2240 read 0x8e7c03f thru 0x8e806f3 (0x46b5 bytes) 2024-09-13T09:18:08.579 INFO:teuthology.orchestra.run.smithi032.stdout:2243 writesame 0xd63e706 thru 0xd649f0e data_size 0x1(0xb809 bytes) 2024-09-13T09:18:08.620 INFO:teuthology.orchestra.run.smithi032.stdout:2244 write 0xec24436 thru 0xec2d496 (0x9061 bytes) 2024-09-13T09:18:08.633 INFO:teuthology.orchestra.run.smithi032.stdout:2245 trunc from 0xec2d497 to 0xb68113f 2024-09-13T09:18:08.649 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:08.642+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:08.653 INFO:teuthology.orchestra.run.smithi032.stdout:2246 writesame 0x9b4bf28 thru 0x9b5b5cd data_size 0x83(0xf6a6 bytes) 2024-09-13T09:18:08.660 INFO:teuthology.orchestra.run.smithi032.stdout:2247 punch from 0x36f317f to 0x36fd6d6, (0xa557 bytes) 2024-09-13T09:18:08.660 INFO:teuthology.orchestra.run.smithi032.stdout:2248 read 0x400fe97 thru 0x401b786 (0xb8f0 bytes) 2024-09-13T09:18:08.660 INFO:teuthology.orchestra.run.smithi032.stdout:2250 read 0x77fcd92 thru 0x7802738 (0x59a7 bytes) 2024-09-13T09:18:08.660 INFO:teuthology.orchestra.run.smithi032.stdout:2251 write 0x6be6725 thru 0x6beb12d (0x4a09 bytes) 2024-09-13T09:18:08.668 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:08.668 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:08.668 INFO:teuthology.orchestra.run.smithi032.stdout:2255 read 0x8d7909f thru 0x8d7b641 (0x25a3 bytes) 2024-09-13T09:18:08.682 INFO:teuthology.orchestra.run.smithi032.stdout:2256 writesame 0xd2df94d thru 0xd2e6aee data_size 0x5(0x71a2 bytes) 2024-09-13T09:18:08.685 INFO:teuthology.orchestra.run.smithi032.stdout:2257 read 0x270e22d thru 0x271d890 (0xf664 bytes) 2024-09-13T09:18:08.686 INFO:teuthology.orchestra.run.smithi032.stdout:2259 write 0x1e6e36d thru 0x1e71ddb (0x3a6f bytes) 2024-09-13T09:18:08.712 INFO:teuthology.orchestra.run.smithi032.stdout:2260 trunc from 0xd2e6aef to 0x41daee7 2024-09-13T09:18:08.730 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:08.730 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:08.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:08.749 INFO:teuthology.orchestra.run.smithi032.stdout:2261 trunc from 0x41daee7 to 0x6871f83 2024-09-13T09:18:08.790 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:08.786+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:08.790 INFO:teuthology.orchestra.run.smithi032.stdout:2262 write 0xeab6907 thru 0xeabe9ae (0x80a8 bytes) 2024-09-13T09:18:08.793 INFO:teuthology.orchestra.run.smithi032.stdout:2263 write 0x86a4043 thru 0x86b0690 (0xc64e bytes) 2024-09-13T09:18:08.811 INFO:teuthology.orchestra.run.smithi032.stdout:2265 write 0xed760ad thru 0xed84339 (0xe28d bytes) 2024-09-13T09:18:08.814 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:08.814 INFO:teuthology.orchestra.run.smithi032.stdout:2270 write 0xd6d097a thru 0xd6de8e1 (0xdf68 bytes) 2024-09-13T09:18:08.833 INFO:teuthology.orchestra.run.smithi032.stdout:2272 trunc from 0xed8433a to 0x930c480 2024-09-13T09:18:08.846 INFO:teuthology.orchestra.run.smithi032.stdout:2273 writesame 0xc8d39 thru 0xd8223 data_size 0x5b(0xf4eb bytes) 2024-09-13T09:18:08.850 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:08.846+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:08.851 INFO:teuthology.orchestra.run.smithi032.stdout:2276 write 0x6576dd0 thru 0x657edd3 (0x8004 bytes) 2024-09-13T09:18:08.859 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:08.871 INFO:teuthology.orchestra.run.smithi032.stdout:2278 write 0x9cdc07b thru 0x9cea61c (0xe5a2 bytes) 2024-09-13T09:18:08.886 INFO:teuthology.orchestra.run.smithi032.stdout:2280 write 0x9d97ccc thru 0x9da13fb (0x9730 bytes) 2024-09-13T09:18:08.899 INFO:teuthology.orchestra.run.smithi032.stdout:2281 trunc from 0x9da13fc to 0x61b0976 2024-09-13T09:18:08.915 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:08.915 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:08.923 INFO:teuthology.orchestra.run.smithi032.stdout:2282 punch from 0x1f07a1f to 0x1f0b912, (0x3ef3 bytes) 2024-09-13T09:18:08.924 INFO:teuthology.orchestra.run.smithi032.stdout:2283 read 0x36b6d03 thru 0x36ba378 (0x3676 bytes) 2024-09-13T09:18:08.924 INFO:teuthology.orchestra.run.smithi032.stdout:2284 punch from 0x11ebdc5 to 0x11f1b0e, (0x5d49 bytes) 2024-09-13T09:18:08.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:08.950 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:08.947+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:08.955 INFO:teuthology.orchestra.run.smithi032.stdout:2288 write 0xd2a2027 thru 0xd2a3dcb (0x1da5 bytes) 2024-09-13T09:18:08.958 INFO:teuthology.orchestra.run.smithi032.stdout:2289 read 0x9619a67 thru 0x962865c (0xebf6 bytes) 2024-09-13T09:18:08.958 INFO:teuthology.orchestra.run.smithi032.stdout:2292 read 0x66e19bb thru 0x66eeceb (0xd331 bytes) 2024-09-13T09:18:08.959 INFO:teuthology.orchestra.run.smithi032.stdout:2293 read 0x9669ee5 thru 0x96747cf (0xa8eb bytes) 2024-09-13T09:18:08.991 INFO:teuthology.orchestra.run.smithi032.stdout:2295 trunc from 0xd2a3dcc to 0x13e6d4b 2024-09-13T09:18:08.997 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:18:08.999 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:18:09.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T09:18:09.036 INFO:teuthology.orchestra.run.smithi032.stdout:2296 write 0x793fbeb thru 0x7945c75 (0x608b bytes) 2024-09-13T09:18:09.040 INFO:teuthology.orchestra.run.smithi032.stdout:2297 read 0x70a2b4f thru 0x70a2e4d (0x2ff bytes) 2024-09-13T09:18:09.040 INFO:teuthology.orchestra.run.smithi032.stdout:2298 read 0x3f4eab2 thru 0x3f52bc4 (0x4113 bytes) 2024-09-13T09:18:09.040 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:09.040 INFO:teuthology.orchestra.run.smithi032.stdout:2300 write 0x304df9b thru 0x30588de (0xa944 bytes) 2024-09-13T09:18:09.043 INFO:teuthology.orchestra.run.smithi032.stdout:2304 write 0x40e56b thru 0x40fd4c (0x17e2 bytes) 2024-09-13T09:18:09.051 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:09.046+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:09.065 INFO:teuthology.orchestra.run.smithi032.stdout:2305 write 0xe5e8d48 thru 0xe5f7355 (0xe60e bytes) 2024-09-13T09:18:09.068 INFO:teuthology.orchestra.run.smithi032.stdout:2306 punch from 0x1825363 to 0x18275fc, (0x2299 bytes) 2024-09-13T09:18:09.089 INFO:teuthology.orchestra.run.smithi032.stdout:2307 trunc from 0xe5f7356 to 0xdf0321 2024-09-13T09:18:09.101 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:09.101 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:09.110 INFO:teuthology.orchestra.run.smithi032.stdout:2308 read 0xa1500f thru 0xa1a62e (0x5620 bytes) 2024-09-13T09:18:09.111 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:09.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:09.119 INFO:teuthology.orchestra.run.smithi032.stdout:2310 write 0x16126f4 thru 0x16145c0 (0x1ecd bytes) 2024-09-13T09:18:09.123 INFO:teuthology.orchestra.run.smithi032.stdout:2311 read 0xb91704 thru 0xb9c40d (0xad0a bytes) 2024-09-13T09:18:09.123 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:09.123 INFO:teuthology.orchestra.run.smithi032.stdout:2314 read 0x79d49c thru 0x7a05a2 (0x3107 bytes) 2024-09-13T09:18:09.134 INFO:teuthology.orchestra.run.smithi032.stdout:2315 write 0x3e011f9 thru 0x3e09df3 (0x8bfb bytes) 2024-09-13T09:18:09.147 INFO:teuthology.orchestra.run.smithi032.stdout:2316 write 0x669aca4 thru 0x669c722 (0x1a7f bytes) 2024-09-13T09:18:09.158 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:09.154+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:09.168 INFO:teuthology.orchestra.run.smithi032.stdout:2317 writesame 0xbdc98be thru 0xbdd62e8 data_size 0xb(0xca2b bytes) 2024-09-13T09:18:09.168 INFO:teuthology.orchestra.run.smithi032.stdout:2318 punch from 0x19ae268 to 0x19bd434, (0xf1cc bytes) 2024-09-13T09:18:09.168 INFO:teuthology.orchestra.run.smithi032.stdout:2319 read 0x41ff5fc thru 0x4207317 (0x7d1c bytes) 2024-09-13T09:18:09.168 INFO:teuthology.orchestra.run.smithi032.stdout:2320 writesame 0x71109a8 thru 0x711a041 data_size 0x25(0x969a bytes) 2024-09-13T09:18:09.172 INFO:teuthology.orchestra.run.smithi032.stdout:2321 read 0xaae720d thru 0xaaebd87 (0x4b7b bytes) 2024-09-13T09:18:09.172 INFO:teuthology.orchestra.run.smithi032.stdout:2322 punch from 0x7eda727 to 0x7ee602c, (0xb905 bytes) 2024-09-13T09:18:09.172 INFO:teuthology.orchestra.run.smithi032.stdout:2323 writesame 0x5cd70a3 thru 0x5ce0878 data_size 0x41(0x97d6 bytes) 2024-09-13T09:18:09.176 INFO:teuthology.orchestra.run.smithi032.stdout:2324 punch from 0x7dbcbe5 to 0x7dca1ec, (0xd607 bytes) 2024-09-13T09:18:09.176 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:09.176 INFO:teuthology.orchestra.run.smithi032.stdout:2327 read 0x15d2cbd thru 0x15d4666 (0x19aa bytes) 2024-09-13T09:18:09.176 INFO:teuthology.orchestra.run.smithi032.stdout:2329 read 0x2abde12 thru 0x2ac44be (0x66ad bytes) 2024-09-13T09:18:09.190 INFO:teuthology.orchestra.run.smithi032.stdout:2330 trunc from 0xbdd62e9 to 0x2b88c41 2024-09-13T09:18:09.207 INFO:teuthology.orchestra.run.smithi032.stdout:2332 punch from 0x13919f9 to 0x139d571, (0xbb78 bytes) 2024-09-13T09:18:09.207 INFO:teuthology.orchestra.run.smithi032.stdout:2334 punch from 0x3741e to 0x41e23, (0xaa05 bytes) 2024-09-13T09:18:09.207 INFO:teuthology.orchestra.run.smithi032.stdout:2335 trunc from 0x2b88c41 to 0x248107e 2024-09-13T09:18:09.214 INFO:teuthology.orchestra.run.smithi032.stdout:2336 read 0x1232b0 thru 0x12834a (0x509b bytes) 2024-09-13T09:18:09.214 INFO:teuthology.orchestra.run.smithi032.stdout:2337 read 0x85a755 thru 0x85b413 (0xcbf bytes) 2024-09-13T09:18:09.215 INFO:teuthology.orchestra.run.smithi032.stdout:2338 read 0x235a711 thru 0x235d64f (0x2f3f bytes) 2024-09-13T09:18:09.215 INFO:teuthology.orchestra.run.smithi032.stdout:2339 writesame 0x1c07fc8 thru 0x1c148f3 data_size 0x7d(0xc92c bytes) 2024-09-13T09:18:09.217 INFO:teuthology.orchestra.run.smithi032.stdout:2341 read 0x1ea3547 thru 0x1ea8745 (0x51ff bytes) 2024-09-13T09:18:09.249 INFO:teuthology.orchestra.run.smithi032.stdout:2342 writesame 0xe91797e thru 0xe9196e1 data_size 0x2c(0x1d64 bytes) 2024-09-13T09:18:09.249 INFO:teuthology.orchestra.run.smithi032.stdout:2344 write 0x51ba4 thru 0x56219 (0x4676 bytes) 2024-09-13T09:18:09.254 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:09.250+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:09.273 INFO:teuthology.orchestra.run.smithi032.stdout:2347 trunc from 0xe9196e2 to 0x2a6ccf8 2024-09-13T09:18:09.287 INFO:teuthology.orchestra.run.smithi032.stdout:2349 trunc from 0x2a6ccf8 to 0x3a24429 2024-09-13T09:18:09.290 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:09.290 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:09.306 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:09.317 INFO:teuthology.orchestra.run.smithi032.stdout:2350 writesame 0x89652e1 thru 0x8973c00 data_size 0x50(0xe920 bytes) 2024-09-13T09:18:09.333 INFO:teuthology.orchestra.run.smithi032.stdout:2351 write 0xb97b60d thru 0xb98581a (0xa20e bytes) 2024-09-13T09:18:09.338 INFO:teuthology.orchestra.run.smithi032.stdout:2352 writesame 0x2be8276 thru 0x2bf622d data_size 0x4(0xdfb8 bytes) 2024-09-13T09:18:09.363 INFO:teuthology.orchestra.run.smithi032.stdout:2353 write 0xc2c8d0c thru 0xc2cb0d7 (0x23cc bytes) 2024-09-13T09:18:09.377 INFO:teuthology.orchestra.run.smithi032.stdout:2355 writesame 0xc3d5059 thru 0xc3e0596 data_size 0x72(0xb53e bytes) 2024-09-13T09:18:09.379 INFO:teuthology.orchestra.run.smithi032.stdout:2356 writesame 0xbb050a2 thru 0xbb0c45e data_size 0x1(0x73bd bytes) 2024-09-13T09:18:09.398 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:09.394+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:09.410 INFO:teuthology.orchestra.run.smithi032.stdout:2357 punch from 0x5cb8998 to 0x5cc49e8, (0xc050 bytes) 2024-09-13T09:18:09.428 INFO:teuthology.orchestra.run.smithi032.stdout:2358 trunc from 0xc3e0597 to 0x1d2faec 2024-09-13T09:18:09.457 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:09.450+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:09.483 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:09.483 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:09.488 INFO:teuthology.orchestra.run.smithi032.stdout:2359 writesame 0xc9164a4 thru 0xc924ffe data_size 0x1(0xeb5b bytes) 2024-09-13T09:18:09.490 INFO:teuthology.orchestra.run.smithi032.stdout:2360 read 0xbb0c804 thru 0xbb174ff (0xacfc bytes) 2024-09-13T09:18:09.491 INFO:teuthology.orchestra.run.smithi032.stdout:2361 write 0xad9baa8 thru 0xada0656 (0x4baf bytes) 2024-09-13T09:18:09.497 INFO:teuthology.orchestra.run.smithi032.stdout:2362 writesame 0x42aa689 thru 0x42b2b81 data_size 0x7(0x84f9 bytes) 2024-09-13T09:18:09.498 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:09.506 INFO:teuthology.orchestra.run.smithi032.stdout:2363 read 0xb12f6c7 thru 0xb13bd4c (0xc686 bytes) 2024-09-13T09:18:09.506 INFO:teuthology.orchestra.run.smithi032.stdout:2365 punch from 0x6938d17 to 0x693ec54, (0x5f3d bytes) 2024-09-13T09:18:09.506 INFO:teuthology.orchestra.run.smithi032.stdout:2366 read 0x70441d1 thru 0x7052199 (0xdfc9 bytes) 2024-09-13T09:18:09.506 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:09.506 INFO:teuthology.orchestra.run.smithi032.stdout:2370 read 0x4867531 thru 0x48756ca (0xe19a bytes) 2024-09-13T09:18:09.507 INFO:teuthology.orchestra.run.smithi032.stdout:2371 punch from 0x47e6c56 to 0x47e7157, (0x501 bytes) 2024-09-13T09:18:09.507 INFO:teuthology.orchestra.run.smithi032.stdout:2375 read 0xb6eb615 thru 0xb6f4099 (0x8a85 bytes) 2024-09-13T09:18:09.507 INFO:teuthology.orchestra.run.smithi032.stdout:2377 write 0x56eccc6 thru 0x56f28d7 (0x5c12 bytes) 2024-09-13T09:18:09.513 INFO:teuthology.orchestra.run.smithi032.stdout:2378 write 0x841e03b thru 0x8427880 (0x9846 bytes) 2024-09-13T09:18:09.522 INFO:teuthology.orchestra.run.smithi032.stdout:2379 trunc from 0xc924fff to 0xccfe2df 2024-09-13T09:18:09.532 INFO:teuthology.orchestra.run.smithi032.stdout:2380 read 0x53ba9e3 thru 0x53bbc10 (0x122e bytes) 2024-09-13T09:18:09.533 INFO:teuthology.orchestra.run.smithi032.stdout:2381 clone 14 order 21 su 1048576 sc 11 2024-09-13T09:18:09.586 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:09.582+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:09.660 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:09.654+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:09.666 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:09.666 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:09.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:09.760 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:09.754+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:09.816 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:09.816 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:09.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:09.874 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:09.875+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:09.950 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:09.951 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:09.962 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:09.963+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:09.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:09.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T09:18:10.035 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T09:18:10.063 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:10.058+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:10.120 INFO:teuthology.orchestra.run.smithi032.stdout:flattening image image_client.0-clone13 2024-09-13T09:18:10.135 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:10.135 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:10.156 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:10.162 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:10.162 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:10.164 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:10.158+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:10.179 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:10.196 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:10.219 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:10.219 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:10.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:10.255 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:10.264 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:10.258+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:10.276 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:10.302 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:10.304 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:10.305 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:10.305 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:10.305 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:10.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:10.324 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:10.325 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:10.341 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-13T09:18:10.348 INFO:teuthology.orchestra.run.smithi032.stdout:2382 writesame 0x65ab582 thru 0x65b789d data_size 0x2(0xc31c bytes) 2024-09-13T09:18:10.365 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:10.363+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:10.369 INFO:teuthology.orchestra.run.smithi032.stdout:2383 punch from 0x1b2f145 to 0x1b32fee, (0x3ea9 bytes) 2024-09-13T09:18:10.369 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:10.369 INFO:teuthology.orchestra.run.smithi032.stdout:2386 read 0xc23e3db thru 0xc24b175 (0xcd9b bytes) 2024-09-13T09:18:10.370 INFO:teuthology.orchestra.run.smithi032.stdout:2390 writesame 0x5a660f8 thru 0x5a68e7d data_size 0x1(0x2d86 bytes) 2024-09-13T09:18:10.375 INFO:teuthology.orchestra.run.smithi032.stdout:2391 read 0x61ea54a thru 0x61f9ddf (0xf896 bytes) 2024-09-13T09:18:10.375 INFO:teuthology.orchestra.run.smithi032.stdout:2392 read 0x6613130 thru 0x6615dfb (0x2ccc bytes) 2024-09-13T09:18:10.376 INFO:teuthology.orchestra.run.smithi032.stdout:2393 punch from 0xadf9262 to 0xadfba94, (0x2832 bytes) 2024-09-13T09:18:10.376 INFO:teuthology.orchestra.run.smithi032.stdout:2394 punch from 0x87f42d7 to 0x87f4650, (0x379 bytes) 2024-09-13T09:18:10.382 INFO:teuthology.orchestra.run.smithi032.stdout:2395 trunc from 0xccfe2df to 0x8e14d64 2024-09-13T09:18:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:2396 punch from 0x37a9f34 to 0x37b6b87, (0xcc53 bytes) 2024-09-13T09:18:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:2397 punch from 0x711e3b2 to 0x71263c3, (0x8011 bytes) 2024-09-13T09:18:10.407 INFO:teuthology.orchestra.run.smithi032.stdout:2399 punch from 0x27dc33a to 0x27df621, (0x32e7 bytes) 2024-09-13T09:18:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:2400 trunc from 0x8e14d64 to 0xf2ddc9 2024-09-13T09:18:10.462 INFO:teuthology.orchestra.run.smithi032.stdout:2401 trunc from 0xf2ddc9 to 0x8d10dcf 2024-09-13T09:18:10.464 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:10.464 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:10.464 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:10.464 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:10.465 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:10.466 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:10.462+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:10.469 INFO:teuthology.orchestra.run.smithi032.stdout:2404 writesame 0x74f7375 thru 0x750277b data_size 0x1(0xb407 bytes) 2024-09-13T09:18:10.483 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:10.503 INFO:teuthology.orchestra.run.smithi032.stdout:2405 trunc from 0x8d10dcf to 0xc905094 2024-09-13T09:18:10.510 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:10.519 INFO:teuthology.orchestra.run.smithi032.stdout:2408 trunc from 0xc905094 to 0x70b881a 2024-09-13T09:18:10.535 INFO:teuthology.orchestra.run.smithi032.stdout:2409 read 0x5b65af3 thru 0x5b69251 (0x375f bytes) 2024-09-13T09:18:10.535 INFO:teuthology.orchestra.run.smithi032.stdout:2411 writesame 0x44c161f thru 0x44ccbdb data_size 0x5(0xb5bd bytes) 2024-09-13T09:18:10.555 INFO:teuthology.orchestra.run.smithi032.stdout:2412 writesame 0xa053397 thru 0xa06063b data_size 0x19(0xd2a5 bytes) 2024-09-13T09:18:10.555 INFO:teuthology.orchestra.run.smithi032.stdout:2414 write 0x2a7663d thru 0x2a80aea (0xa4ae bytes) 2024-09-13T09:18:10.560 INFO:teuthology.orchestra.run.smithi032.stdout:2415 writesame 0x58cac47 thru 0x58ce85b data_size 0x3(0x3c15 bytes) 2024-09-13T09:18:10.566 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:10.567+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:10.567 INFO:teuthology.orchestra.run.smithi032.stdout:2417 write 0x28d6a84 thru 0x28de3e4 (0x7961 bytes) 2024-09-13T09:18:10.572 INFO:teuthology.orchestra.run.smithi032.stdout:2420 read 0x198a897 thru 0x19942c8 (0x9a32 bytes) 2024-09-13T09:18:10.572 INFO:teuthology.orchestra.run.smithi032.stdout:2422 read 0x7b9b638 thru 0x7ba777a (0xc143 bytes) 2024-09-13T09:18:10.582 INFO:teuthology.orchestra.run.smithi032.stdout:2423 write 0xbb4e927 thru 0xbb4ef9e (0x678 bytes) 2024-09-13T09:18:10.590 INFO:teuthology.orchestra.run.smithi032.stdout:2424 trunc from 0xbb4ef9f to 0x9103203 2024-09-13T09:18:10.606 INFO:teuthology.orchestra.run.smithi032.stdout:2425 punch from 0x6f6d439 to 0x6f7c6ba, (0xf281 bytes) 2024-09-13T09:18:10.606 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:10.606 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:10.607 INFO:teuthology.orchestra.run.smithi032.stdout:2430 flatten 2024-09-13T09:18:10.627 INFO:teuthology.orchestra.run.smithi032.stdout:2432 write 0xa37d04 thru 0xa3e25c (0x6559 bytes) 2024-09-13T09:18:10.630 INFO:teuthology.orchestra.run.smithi032.stdout:2434 read 0x5862df3 thru 0x586444c (0x165a bytes) 2024-09-13T09:18:10.632 INFO:teuthology.orchestra.run.smithi032.stdout:2435 read 0x1a7c862 thru 0x1a88428 (0xbbc7 bytes) 2024-09-13T09:18:10.632 INFO:teuthology.orchestra.run.smithi032.stdout:2436 read 0x1e6ee77 thru 0x1e6f5e0 (0x76a bytes) 2024-09-13T09:18:10.632 INFO:teuthology.orchestra.run.smithi032.stdout:2437 read 0x24b470f thru 0x24bdd0f (0x9601 bytes) 2024-09-13T09:18:10.632 INFO:teuthology.orchestra.run.smithi032.stdout:2438 writesame 0x51c4b36 thru 0x51c6718 data_size 0xb(0x1be3 bytes) 2024-09-13T09:18:10.637 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5479 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.098122774, 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027937460000000001, 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5479 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539250+0000", 2024-09-13T09:18:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539256+0000", 2024-09-13T09:18:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:18:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539263+0000", 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3779999999999998e-06 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539273+0000", 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0457e-05 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567010+0000", 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027736878 2024-09-13T09:18:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567083+0000", 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3187000000000002e-05 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567188+0000", 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010491499999999999 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5480 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.098095556000001, 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028132641999999999, 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5480 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539280+0000", 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539297+0000", 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7147999999999999e-05 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539303+0000", 2024-09-13T09:18:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9039999999999997e-06 2024-09-13T09:18:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567230+0000", 2024-09-13T09:18:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027927250000000001 2024-09-13T09:18:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567316+0000", 2024-09-13T09:18:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5970999999999994e-05 2024-09-13T09:18:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567411+0000", 2024-09-13T09:18:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4523000000000003e-05 2024-09-13T09:18:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5481 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.098065559999998, 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028317978000000001, 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5481 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539307+0000", 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539311+0000", 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.264e-06 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539317+0000", 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539322+0000", 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6949999999999999e-06 2024-09-13T09:18:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567446+0000", 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028124341000000001 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567527+0000", 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0785000000000007e-05 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567626+0000", 2024-09-13T09:18:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9103000000000003e-05 2024-09-13T09:18:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5482 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:18:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.098047735000002, 2024-09-13T09:18:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028619326, 2024-09-13T09:18:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5482 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539325+0000", 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539333+0000", 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.055e-06 2024-09-13T09:18:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539340+0000", 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4980000000000003e-06 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539344+0000", 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5369999999999999e-06 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567655+0000", 2024-09-13T09:18:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028310906 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567732+0000", 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6829999999999995e-05 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567945+0000", 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000212909 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5483 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.098024473999999, 2024-09-13T09:18:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028784015999999999, 2024-09-13T09:18:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5483 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539351+0000", 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539357+0000", 2024-09-13T09:18:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0090000000000004e-06 2024-09-13T09:18:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539361+0000", 2024-09-13T09:18:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.357e-06 2024-09-13T09:18:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567974+0000", 2024-09-13T09:18:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028612385000000001 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568050+0000", 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5820000000000003e-05 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568133+0000", 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3551e-05 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5484 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:18:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.098007922000001, 2024-09-13T09:18:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028949923999999998, 2024-09-13T09:18:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5484 2024-09-13T09:18:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539365+0000", 2024-09-13T09:18:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539367+0000", 2024-09-13T09:18:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:18:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539374+0000", 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539378+0000", 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568159+0000", 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028780490999999998 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568236+0000", 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7156000000000005e-05 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568316+0000", 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9598999999999995e-05 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5485 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.097188042999999, 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028332593, 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5485 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540189+0000", 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540198+0000", 2024-09-13T09:18:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9099999999999994e-06 2024-09-13T09:18:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540206+0000", 2024-09-13T09:18:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-13T09:18:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568360+0000", 2024-09-13T09:18:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028154324000000001 2024-09-13T09:18:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568436+0000", 2024-09-13T09:18:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5821000000000005e-05 2024-09-13T09:18:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568518+0000", 2024-09-13T09:18:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1895000000000001e-05 2024-09-13T09:18:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5486 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.097135034000001, 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028479536999999999, 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5486 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540241+0000", 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4290000000000001e-06 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540249+0000", 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9480000000000008e-06 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540255+0000", 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6019999999999999e-06 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568543+0000", 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028288260999999999 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568636+0000", 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2627999999999995e-05 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568718+0000", 2024-09-13T09:18:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2213999999999996e-05 2024-09-13T09:18:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5487 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:18:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.097113949000001, 2024-09-13T09:18:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028653435000000001, 2024-09-13T09:18:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5487 2024-09-13T09:18:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:18:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:18:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540259+0000", 2024-09-13T09:18:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540262+0000", 2024-09-13T09:18:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.413e-06 2024-09-13T09:18:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540268+0000", 2024-09-13T09:18:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3940000000000001e-06 2024-09-13T09:18:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540272+0000", 2024-09-13T09:18:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3989999999999997e-06 2024-09-13T09:18:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568745+0000", 2024-09-13T09:18:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028472173999999999 2024-09-13T09:18:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568822+0000", 2024-09-13T09:18:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7694000000000005e-05 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568913+0000", 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0779000000000006e-05 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5488 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 430080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.097096549, 2024-09-13T09:18:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028788326999999999, 2024-09-13T09:18:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5488 2024-09-13T09:18:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540279+0000", 2024-09-13T09:18:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2409999999999998e-06 2024-09-13T09:18:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540285+0000", 2024-09-13T09:18:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3180000000000003e-06 2024-09-13T09:18:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540300+0000", 2024-09-13T09:18:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4625e-05 2024-09-13T09:18:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568939+0000", 2024-09-13T09:18:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028639688 2024-09-13T09:18:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569016+0000", 2024-09-13T09:18:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6556000000000004e-05 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569065+0000", 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9246000000000003e-05 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5489 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:18:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.097068955000001, 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028875332, 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5489 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540304+0000", 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540307+0000", 2024-09-13T09:18:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4370000000000001e-06 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540313+0000", 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540318+0000", 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569079+0000", 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028760714 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569128+0000", 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8702999999999998e-05 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569180+0000", 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2105999999999999e-05 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5490 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.097050459999998, 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029041027000000001, 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5490 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540326+0000", 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9840000000000001e-06 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540332+0000", 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9730000000000002e-06 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540338+0000", 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3550000000000001e-06 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569202+0000", 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028863521 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569276+0000", 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569364+0000", 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7594000000000002e-05 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5491 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.097031076, 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029221864, 2024-09-13T09:18:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5491 2024-09-13T09:18:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540350+0000", 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4250000000000001e-06 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540356+0000", 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2960000000000004e-06 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540361+0000", 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569388+0000", 2024-09-13T09:18:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029026744 2024-09-13T09:18:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569474+0000", 2024-09-13T09:18:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6519000000000003e-05 2024-09-13T09:18:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569564+0000", 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9969999999999994e-05 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5492 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.097007778999998, 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029401203000000001, 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5492 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540365+0000", 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540368+0000", 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540374+0000", 2024-09-13T09:18:10.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9349999999999999e-06 2024-09-13T09:18:10.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540378+0000", 2024-09-13T09:18:10.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.651e-06 2024-09-13T09:18:10.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569591+0000", 2024-09-13T09:18:10.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029212557 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569668+0000", 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569767+0000", 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9061000000000006e-05 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5493 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.096990914999999, 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029599424999999999, 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5493 2024-09-13T09:18:10.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.706 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:18:10.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:18:10.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540382+0000", 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540384+0000", 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0430000000000002e-06 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540390+0000", 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:18:10.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540395+0000", 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5990000000000004e-06 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569795+0000", 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029400582000000001 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569880+0000", 2024-09-13T09:18:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5210999999999994e-05 2024-09-13T09:18:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569982+0000", 2024-09-13T09:18:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101584 2024-09-13T09:18:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.709 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5494 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.096974830000001, 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029770637999999999, 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5494 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:18:10.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:18:10.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540398+0000", 2024-09-13T09:18:10.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540406+0000", 2024-09-13T09:18:10.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:18:10.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540413+0000", 2024-09-13T09:18:10.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:18:10.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540418+0000", 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8289999999999998e-06 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570007+0000", 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029588851999999999 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570084+0000", 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7253e-05 2024-09-13T09:18:10.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570169+0000", 2024-09-13T09:18:10.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5391999999999998e-05 2024-09-13T09:18:10.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.714 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.714 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5495 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.714 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:10.714 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.096936513999999, 2024-09-13T09:18:10.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029938664, 2024-09-13T09:18:10.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.715 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.715 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5495 2024-09-13T09:18:10.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.715 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:10.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:10.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:10.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540440+0000", 2024-09-13T09:18:10.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0319999999999999e-06 2024-09-13T09:18:10.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540446+0000", 2024-09-13T09:18:10.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8549999999999996e-06 2024-09-13T09:18:10.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540451+0000", 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570198+0000", 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029746060000000001 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570284+0000", 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6044999999999994e-05 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570376+0000", 2024-09-13T09:18:10.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2071000000000007e-05 2024-09-13T09:18:10.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.719 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.719 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5496 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.719 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:10.719 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.096917559000001, 2024-09-13T09:18:10.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030121617999999999, 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5496 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:10.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:10.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540458+0000", 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540464+0000", 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.156e-06 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540468+0000", 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:18:10.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570404+0000", 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029935632 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570488+0000", 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3873000000000001e-05 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570578+0000", 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9757000000000001e-05 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5497 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.096900994999999, 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030305259000000001, 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:10.724 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5497 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540472+0000", 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540474+0000", 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0490000000000002e-06 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540480+0000", 2024-09-13T09:18:10.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1970000000000001e-06 2024-09-13T09:18:10.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540491+0000", 2024-09-13T09:18:10.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0964e-05 2024-09-13T09:18:10.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570608+0000", 2024-09-13T09:18:10.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030116105000000001 2024-09-13T09:18:10.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570688+0000", 2024-09-13T09:18:10.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0143999999999997e-05 2024-09-13T09:18:10.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570778+0000", 2024-09-13T09:18:10.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0131999999999997e-05 2024-09-13T09:18:10.728 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.5633689650000004, 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:10.730 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:18:10.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.730 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:10.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:10.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:18:10.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:10.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:18:10.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:10.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:18:10.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:18:10.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:10.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:18:10.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:18:10.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:18:10.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:10.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:18:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:18:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:18:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:18:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:18:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:18:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:18:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:10.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:18:10.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:18:10.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:10.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:18:10.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:18:10.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:10.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:10.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:10.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:18:10.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:18:10.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.736 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:10.736 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:10.736 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:10.737 INFO:teuthology.orchestra.run.smithi032.stdout:2439 clone 15 order 22 su 1048576 sc 12 2024-09-13T09:18:10.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:10.739 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:10.734+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:10.838 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:10.834+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:10.929 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:10.939 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:10.934+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T09:18:11.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:18:11.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:18:11.040 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:11.034+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:11.057 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:11.057 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:11.058 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:11.058 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:11.058 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:11.077 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:11.170 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:11.171+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:11.242 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:11.238+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:11.254 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:11.254 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:11.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:11.254 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:11.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.254 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "pg_update_log_missing(3.4 epoch 35/22 rep_tid 113 entries 35'31 (0'0) error 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head by client.4485.0:3311 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 31'29)", 2024-09-13T09:18:11.254 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:18:11.254 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.726074130000001, 2024-09-13T09:18:11.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033204409999999999, 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527920+0000", 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527918+0000", 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527943+0000", 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4752999999999998e-05 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.527988+0000", 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4693000000000001e-05 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528029+0000", 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1155999999999997e-05 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.528194+0000", 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164792 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.531241+0000", 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003046867 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3323 3.4 e35/22 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head v 35'34, mlcod=35'31)", 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.718199204000001, 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037621640000000001, 2024-09-13T09:18:11.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.259 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:18:11.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:18:11.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535793+0000", 2024-09-13T09:18:11.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:11.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535811+0000", 2024-09-13T09:18:11.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8170000000000001e-05 2024-09-13T09:18:11.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535840+0000", 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9105999999999999e-05 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535877+0000", 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7017000000000002e-05 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536001+0000", 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012342899999999999 2024-09-13T09:18:11.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536073+0000", 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2123e-05 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539433+0000", 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033596630000000001 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539557+0000", 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012470600000000001 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:15977 3.4 3:27c24f46:::rbd_data.11856af9a4d3.0000000000000005:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.258929690999999, 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003633163, 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.263 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.264 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.264 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.264 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15977 2024-09-13T09:18:11.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.264 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:18:11.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995065+0000", 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995063+0000", 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995071+0000", 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3930000000000002e-06 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995086+0000", 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4970999999999999e-05 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995095+0000", 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8810000000000002e-06 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995180+0000", 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5012000000000005e-05 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.995378+0000", 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019788600000000001 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:51.998698+0000", 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003320267 2024-09-13T09:18:11.267 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.268 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.268 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16010 3.4 3:27c24f46:::rbd_data.11856af9a4d3.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:11.268 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:18:11.268 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.253320898999998, 2024-09-13T09:18:11.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037539579999999999, 2024-09-13T09:18:11.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.268 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.268 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16010 2024-09-13T09:18:11.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.269 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:18:11.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000672+0000", 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000679+0000", 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9120000000000001e-06 2024-09-13T09:18:11.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000693+0000", 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3745e-05 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000702+0000", 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.064e-06 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000763+0000", 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1309000000000005e-05 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000877+0000", 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114214 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.001145+0000", 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000267765 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.002514+0000", 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013688730000000001 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004301+0000", 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017869839999999999 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004380+0000", 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8861000000000003e-05 2024-09-13T09:18:11.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004427+0000", 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7713000000000002e-05 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16047 3.0 3:002d813e:::rbd_data.11856af9a4d3.0000000000000014:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.244443897, 2024-09-13T09:18:11.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030667929999999999, 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16047 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009550+0000", 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009548+0000", 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009559+0000", 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0936e-05 2024-09-13T09:18:11.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009579+0000", 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9976e-05 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009590+0000", 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1228e-05 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009670+0000", 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9765999999999995e-05 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009776+0000", 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106049 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.009991+0000", 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021494999999999999 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.011621+0000", 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016294650000000001 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012469+0000", 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000848598 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012551+0000", 2024-09-13T09:18:11.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2144999999999994e-05 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.012617+0000", 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5827999999999994e-05 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16226 3.4 3:22419c10:::rbd_data.1185f20bdb42.0000000000000013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:18:11.280 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.025443793000001, 2024-09-13T09:18:11.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033492510000000001, 2024-09-13T09:18:11.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.281 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.281 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.281 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.281 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.281 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16226 2024-09-13T09:18:11.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.281 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228551+0000", 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228546+0000", 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999957 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228569+0000", 2024-09-13T09:18:11.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2347000000000001e-05 2024-09-13T09:18:11.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228618+0000", 2024-09-13T09:18:11.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9509999999999999e-05 2024-09-13T09:18:11.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228643+0000", 2024-09-13T09:18:11.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4759000000000001e-05 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228738+0000", 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5273999999999996e-05 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228819+0000", 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0623000000000004e-05 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:11.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.228921+0000", 2024-09-13T09:18:11.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010215199999999999 2024-09-13T09:18:11.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.230000+0000", 2024-09-13T09:18:11.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010793510000000001 2024-09-13T09:18:11.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231718+0000", 2024-09-13T09:18:11.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001717873 2024-09-13T09:18:11.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231813+0000", 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4473000000000002e-05 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.231900+0000", 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7158999999999997e-05 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16293 3.4 3:22419c10:::rbd_data.1185f20bdb42.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:18:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.871833112000001, 2024-09-13T09:18:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015655326000000001, 2024-09-13T09:18:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16293 2024-09-13T09:18:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382158+0000", 2024-09-13T09:18:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382176+0000", 2024-09-13T09:18:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7863000000000002e-05 2024-09-13T09:18:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382211+0000", 2024-09-13T09:18:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.481e-05 2024-09-13T09:18:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382226+0000", 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.537e-05 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382354+0000", 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012803599999999999 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382591+0000", 2024-09-13T09:18:11.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023629099999999999 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.385269+0000", 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002678532 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397515+0000", 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012245946000000001 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397551+0000", 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5856999999999999e-05 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397699+0000", 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148255 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397817+0000", 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117299 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.292 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4485.0:16293 3.4 e42/41 ondisk, result = 0)", 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.861501572000002, 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0050343319999999999, 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:18:11.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392491+0000", 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392500+0000", 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8300000000000002e-06 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392529+0000", 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9260000000000001e-05 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392542+0000", 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2969999999999999e-05 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397468+0000", 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0049256880000000001 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397496+0000", 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8209999999999999e-05 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397527+0000", 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1276999999999999e-05 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.298 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16837 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [call rbd.sparse_copyup in=173207b,set-alloc-hint object_size 16777216 write_size 16777216,write 3145728~10909 in=10909b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.993870759, 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004073887, 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16837 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260121+0000", 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:11.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260266+0000", 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014495999999999999 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260287+0000", 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1336e-05 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260296+0000", 2024-09-13T09:18:11.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6170000000000007e-06 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260352+0000", 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6252999999999998e-05 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260520+0000", 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000168381 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:18:11.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260807+0000", 2024-09-13T09:18:11.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028624100000000001 2024-09-13T09:18:11.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.262062+0000", 2024-09-13T09:18:11.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012551490000000001 2024-09-13T09:18:11.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264128+0000", 2024-09-13T09:18:11.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020660380000000001 2024-09-13T09:18:11.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264167+0000", 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9714e-05 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264198+0000", 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0105000000000001e-05 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:17005 3.1a e43/41 3:5e94b56b:::rbd_data.11859d3d11dd.0000000000000008:head v 43'54, mlcod=42'52)", 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.868306101999998, 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031754880000000002, 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.385688+0000", 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.386997+0000", 2024-09-13T09:18:11.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00130933 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387013+0000", 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5752000000000001e-05 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387022+0000", 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1689999999999998e-06 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387092+0000", 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9385000000000001e-05 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.387127+0000", 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5061000000000002e-05 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.388821+0000", 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001694463 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.388864+0000", 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2721e-05 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:17028 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [stat,zero 4194304~4718592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.844487649000001, 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036807770000000001, 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.309 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 17028 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409507+0000", 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409504+0000", 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409515+0000", 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0485e-05 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409540+0000", 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4808e-05 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409550+0000", 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0376e-05 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409616+0000", 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6130999999999994e-05 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.409725+0000", 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108614 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.413188+0000", 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003462849 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18757 3.4 3:21c7a0ac:::rbd_data.1185b72576e4.0000000000000074:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.623959957, 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066789190000000002, 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18757 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630032+0000", 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630044+0000", 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2245999999999999e-05 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630067+0000", 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3404000000000002e-05 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630078+0000", 2024-09-13T09:18:11.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1158999999999999e-05 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630156+0000", 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7520000000000003e-05 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630357+0000", 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020064700000000001 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:11.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.631878+0000", 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015212940000000001 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636574+0000", 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046960719999999999 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636612+0000", 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7499999999999997e-05 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636669+0000", 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7286e-05 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636713+0000", 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4530000000000002e-05 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18822 3.4 3:250382e8:::rbd_data.1185c2de398f.0000000000000005:e [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.605530871999999, 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031435460000000001, 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18822 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:18:11.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:18:11.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648463+0000", 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648464+0000", 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-06 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648468+0000", 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1999999999999999e-06 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.648470+0000", 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7389999999999999e-06 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651484+0000", 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003014128 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651541+0000", 2024-09-13T09:18:11.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6304999999999997e-05 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.651607+0000", 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6280999999999998e-05 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19013 3.4 3:21c7a0ac:::rbd_data.1185b72576e4.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.874194+0000", 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.379800413, 2024-09-13T09:18:11.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033037850000000001, 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19013 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874194+0000", 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874194+0000", 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874191+0000", 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874207+0000", 2024-09-13T09:18:11.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6441000000000001e-05 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874235+0000", 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7554000000000001e-05 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874246+0000", 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1114999999999999e-05 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874331+0000", 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4608e-05 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874450+0000", 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011943900000000001 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.874607+0000", 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015705600000000001 2024-09-13T09:18:11.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.875492+0000", 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00088451199999999995 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.877353+0000", 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018616220000000001 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.877437+0000", 2024-09-13T09:18:11.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3698999999999999e-05 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.877498+0000", 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0745999999999997e-05 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:20630 3.4 3:23dc35fd:::rbd_data.1185b72576e4.000000000000007c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:00.986895+0000", 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.267099534, 2024-09-13T09:18:11.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0040477539999999998, 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 20630 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986895+0000", 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986895+0000", 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986893+0000", 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986899+0000", 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-13T09:18:11.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986910+0000", 2024-09-13T09:18:11.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1105e-05 2024-09-13T09:18:11.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986917+0000", 2024-09-13T09:18:11.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:18:11.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986963+0000", 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5932e-05 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.987020+0000", 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6858999999999999e-05 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.987104+0000", 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4250000000000001e-05 2024-09-13T09:18:11.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.987786+0000", 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00068249499999999998 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.990828+0000", 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030412099999999999 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.990893+0000", 2024-09-13T09:18:11.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5117999999999996e-05 2024-09-13T09:18:11.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.990943+0000", 2024-09-13T09:18:11.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9789000000000001e-05 2024-09-13T09:18:11.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.336 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.336 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:21345 3.0 3:004c516c:::rbd_header.1185d6f8ec87:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:18:11.336 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:02.094060+0000", 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.1599345579999998, 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004611782, 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 21345 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094060+0000", 2024-09-13T09:18:11.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094060+0000", 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094057+0000", 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094070+0000", 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2787e-05 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094095+0000", 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4873000000000001e-05 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094106+0000", 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0954e-05 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094171+0000", 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4958999999999999e-05 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094344+0000", 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000173098 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094590+0000", 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024549199999999999 2024-09-13T09:18:11.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.095579+0000", 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00098965800000000003 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.098585+0000", 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030058390000000002 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.098630+0000", 2024-09-13T09:18:11.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5393999999999999e-05 2024-09-13T09:18:11.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.098672+0000", 2024-09-13T09:18:11.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1153999999999999e-05 2024-09-13T09:18:11.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.342 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.342 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22933 3.6 3:635a9857:::rbd_data.1185d6f8ec87.0000000000000005:head [write 3068480~25003] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:18:11.342 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:07.550381+0000", 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.7036134270000001, 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003102559, 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22933 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.550381+0000", 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.550381+0000", 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.550378+0000", 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.550439+0000", 2024-09-13T09:18:11.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1072000000000001e-05 2024-09-13T09:18:11.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.550472+0000", 2024-09-13T09:18:11.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2960000000000003e-05 2024-09-13T09:18:11.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.550487+0000", 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4956e-05 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.550598+0000", 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110578 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.550780+0000", 2024-09-13T09:18:11.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018255800000000001 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.550988+0000", 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000207794 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.552823+0000", 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001834965 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.553351+0000", 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00052831700000000004 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.553429+0000", 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7737999999999993e-05 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:07.553484+0000", 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4330999999999999e-05 2024-09-13T09:18:11.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.349 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.349 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:24733 3.6 3:673d20ef:::rbd_data.118513c288ae.000000000000002e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-13T09:18:11.349 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:10.357682+0000", 2024-09-13T09:18:11.349 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.89631195600000002, 2024-09-13T09:18:11.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013452253000000001, 2024-09-13T09:18:11.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 24733 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357682+0000", 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357682+0000", 2024-09-13T09:18:11.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357681+0000", 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357690+0000", 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357705+0000", 2024-09-13T09:18:11.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4847e-05 2024-09-13T09:18:11.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357713+0000", 2024-09-13T09:18:11.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.469e-06 2024-09-13T09:18:11.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357776+0000", 2024-09-13T09:18:11.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3490000000000004e-05 2024-09-13T09:18:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357937+0000", 2024-09-13T09:18:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161077 2024-09-13T09:18:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:18:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.360165+0000", 2024-09-13T09:18:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022276330000000001 2024-09-13T09:18:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.371006+0000", 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010841046999999999 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.371023+0000", 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6872e-05 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.371083+0000", 2024-09-13T09:18:11.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9991999999999998e-05 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.371135+0000", 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2064000000000002e-05 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4485.0:24733 3.6 e59/52 ondisk, result = 0)", 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:10.366395+0000", 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.88759979499999997, 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046156160000000003, 2024-09-13T09:18:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.366395+0000", 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.366395+0000", 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.366394+0000", 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.366398+0000", 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.335e-06 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.366412+0000", 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3879e-05 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.366419+0000", 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9319999999999999e-06 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.370987+0000", 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045677410000000002 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.370999+0000", 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1884e-05 2024-09-13T09:18:11.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.371010+0000", 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:24735 3.6 3:673d20ef:::rbd_data.118513c288ae.000000000000002e:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:10.372012+0000", 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.88198222999999998, 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0053273979999999997, 2024-09-13T09:18:11.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 24735 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372012+0000", 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372012+0000", 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372011+0000", 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372017+0000", 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.331e-06 2024-09-13T09:18:11.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372030+0000", 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2203e-05 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372036+0000", 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2999999999999998e-06 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372071+0000", 2024-09-13T09:18:11.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.54e-05 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372124+0000", 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2621000000000003e-05 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372908+0000", 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00078428899999999997 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.377224+0000", 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0043153510000000003 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.377238+0000", 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4114000000000001e-05 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.377291+0000", 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3823999999999998e-05 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.377340+0000", 2024-09-13T09:18:11.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8115999999999998e-05 2024-09-13T09:18:11.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.365 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:11.366 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-13T09:18:11.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:11.367 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:11.367+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:11.467 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:11.467+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:11.535 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:11.535 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:11.535 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:11.535 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:11.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:11.594 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:11.595+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:11.612 INFO:teuthology.orchestra.run.smithi032.stdout:2440 read 0x668a460 thru 0x668b220 (0xdc1 bytes) 2024-09-13T09:18:11.612 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:11.612 INFO:teuthology.orchestra.run.smithi032.stdout:2442 write 0x1dfd860 thru 0x1e02e22 (0x55c3 bytes) 2024-09-13T09:18:11.628 INFO:teuthology.orchestra.run.smithi032.stdout:2443 writesame 0x934032 thru 0x943b27 data_size 0x5b(0xfaf6 bytes) 2024-09-13T09:18:11.635 INFO:teuthology.orchestra.run.smithi032.stdout:2445 read 0x6d1f57e thru 0x6d25096 (0x5b19 bytes) 2024-09-13T09:18:11.635 INFO:teuthology.orchestra.run.smithi032.stdout:2446 write 0x8bd8dbe thru 0x8be5e7c (0xd0bf bytes) 2024-09-13T09:18:11.641 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:11.641 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:11.641 INFO:teuthology.orchestra.run.smithi032.stdout:2449 read 0x2b19cd thru 0x2b4313 (0x2947 bytes) 2024-09-13T09:18:11.641 INFO:teuthology.orchestra.run.smithi032.stdout:2453 punch from 0x66c14a5 to 0x66c65cf, (0x512a bytes) 2024-09-13T09:18:11.641 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:11.641 INFO:teuthology.orchestra.run.smithi032.stdout:2455 punch from 0x66b2967 to 0x66b6636, (0x3ccf bytes) 2024-09-13T09:18:11.658 INFO:teuthology.orchestra.run.smithi032.stdout:2456 write 0xe4c1b43 thru 0xe4c443b (0x28f9 bytes) 2024-09-13T09:18:11.664 INFO:teuthology.orchestra.run.smithi032.stdout:2457 write 0x3d3cf0 thru 0x3dd0b3 (0x93c4 bytes) 2024-09-13T09:18:11.668 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:11.667+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:2458 write 0x97d7579 thru 0x97e17d8 (0xa260 bytes) 2024-09-13T09:18:11.677 INFO:teuthology.orchestra.run.smithi032.stdout:2459 write 0x2e48df1 thru 0x2e4e152 (0x5362 bytes) 2024-09-13T09:18:11.683 INFO:teuthology.orchestra.run.smithi032.stdout:2460 write 0xa8e428 thru 0xa91ca9 (0x3882 bytes) 2024-09-13T09:18:11.685 INFO:teuthology.orchestra.run.smithi032.stdout:2461 clone 16 order 20 su 131072 sc 10 2024-09-13T09:18:11.718 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:11.719 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:11.719 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:11.732 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:11.770 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:11.766+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:11.871 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:11.871+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:11.903 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:11.903 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:11.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:11.903 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:11.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.904 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:11.904 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:11.904 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.365119176, 2024-09-13T09:18:11.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:18:11.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:11.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:18:11.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:18:11.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:18:11.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:18:11.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:18:11.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:18:11.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:18:11.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:18:11.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:11.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:18:11.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:18:11.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:18:11.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19325 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2514944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:11.910 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971642510000001, 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041428072000000003, 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19325 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:11.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929839+0000", 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929845+0000", 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9510000000000003e-06 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929850+0000", 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971169+0000", 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041318743999999998 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971214+0000", 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971265+0000", 2024-09-13T09:18:11.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0963e-05 2024-09-13T09:18:11.915 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.915 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.915 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.915 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19326 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2613248~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.915 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:11.915 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971625574000001, 2024-09-13T09:18:11.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041534459000000003, 2024-09-13T09:18:11.915 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.915 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.916 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.916 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.916 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.916 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19326 2024-09-13T09:18:11.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.916 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:11.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:11.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:11.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929856+0000", 2024-09-13T09:18:11.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0470000000000001e-06 2024-09-13T09:18:11.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929862+0000", 2024-09-13T09:18:11.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3300000000000004e-06 2024-09-13T09:18:11.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929867+0000", 2024-09-13T09:18:11.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:18:11.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971293+0000", 2024-09-13T09:18:11.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041425785999999999 2024-09-13T09:18:11.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971337+0000", 2024-09-13T09:18:11.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4240999999999999e-05 2024-09-13T09:18:11.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971389+0000", 2024-09-13T09:18:11.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1483999999999997e-05 2024-09-13T09:18:11.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.920 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19327 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2711552~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971608507999999, 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041642875000000003, 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19327 2024-09-13T09:18:11.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:11.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929873+0000", 2024-09-13T09:18:11.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.26e-06 2024-09-13T09:18:11.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929879+0000", 2024-09-13T09:18:11.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0519999999999998e-06 2024-09-13T09:18:11.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929884+0000", 2024-09-13T09:18:11.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:18:11.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971416+0000", 2024-09-13T09:18:11.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041532122999999997 2024-09-13T09:18:11.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971460+0000", 2024-09-13T09:18:11.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4456999999999998e-05 2024-09-13T09:18:11.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971514+0000", 2024-09-13T09:18:11.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3579000000000003e-05 2024-09-13T09:18:11.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19328 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2809856~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971591745, 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041750662000000001, 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19328 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929890+0000", 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929896+0000", 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929901+0000", 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971542+0000", 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041640442 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971587+0000", 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4972999999999998e-05 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971639+0000", 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1990000000000002e-05 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19329 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2908160~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971549753, 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041833958999999997, 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19329 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:18:11.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:18:11.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929929+0000", 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929932+0000", 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0699999999999998e-06 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929941+0000", 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.149e-06 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929947+0000", 2024-09-13T09:18:11.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.699e-06 2024-09-13T09:18:11.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971666+0000", 2024-09-13T09:18:11.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041719224999999999 2024-09-13T09:18:11.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971711+0000", 2024-09-13T09:18:11.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4823000000000001e-05 2024-09-13T09:18:11.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971764+0000", 2024-09-13T09:18:11.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2741999999999997e-05 2024-09-13T09:18:11.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.935 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.935 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19330 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 3006464~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971527011999999, 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041949980999999997, 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19330 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929952+0000", 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929955+0000", 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929961+0000", 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1199999999999999e-06 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929966+0000", 2024-09-13T09:18:11.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971792+0000", 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041826680999999998 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971838+0000", 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5939000000000002e-05 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971903+0000", 2024-09-13T09:18:11.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4499999999999996e-05 2024-09-13T09:18:11.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.940 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19331 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 3104768~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.940 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:18:11.940 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971510045, 2024-09-13T09:18:11.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042058848000000003, 2024-09-13T09:18:11.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19331 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929969+0000", 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929975+0000", 2024-09-13T09:18:11.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.434e-06 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929981+0000", 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929986+0000", 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1780000000000002e-06 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971931+0000", 2024-09-13T09:18:11.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041944417999999997 2024-09-13T09:18:11.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971976+0000", 2024-09-13T09:18:11.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4962000000000001e-05 2024-09-13T09:18:11.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972029+0000", 2024-09-13T09:18:11.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2768e-05 2024-09-13T09:18:11.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.945 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.945 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19332 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 65536~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.945 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:18:11.945 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971482875, 2024-09-13T09:18:11.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042158568, 2024-09-13T09:18:11.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19332 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:18:11.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929996+0000", 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929999+0000", 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.306e-06 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930005+0000", 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0379999999999996e-06 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930010+0000", 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0969999999999996e-06 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972057+0000", 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042047117000000002 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972102+0000", 2024-09-13T09:18:11.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5422e-05 2024-09-13T09:18:11.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972155+0000", 2024-09-13T09:18:11.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2964999999999999e-05 2024-09-13T09:18:11.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.949 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19333 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 163840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971465784999999, 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042266805999999997, 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19333 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.950 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:11.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930016+0000", 2024-09-13T09:18:11.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:18:11.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930022+0000", 2024-09-13T09:18:11.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2550000000000003e-06 2024-09-13T09:18:11.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930027+0000", 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8060000000000002e-06 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972183+0000", 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042156276999999999 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972228+0000", 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5317e-05 2024-09-13T09:18:11.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972281+0000", 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2398999999999998e-05 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19334 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 262144~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971447940000001, 2024-09-13T09:18:11.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042375573, 2024-09-13T09:18:11.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.955 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19334 2024-09-13T09:18:11.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.955 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:18:11.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930031+0000", 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930034+0000", 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930040+0000", 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0789999999999997e-06 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930045+0000", 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0139999999999998e-06 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972307+0000", 2024-09-13T09:18:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042262685000000001 2024-09-13T09:18:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972352+0000", 2024-09-13T09:18:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5021000000000001e-05 2024-09-13T09:18:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972407+0000", 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4945000000000003e-05 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19335 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 360448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971431126000001, 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042483785000000003, 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19335 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:18:11.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930048+0000", 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930050+0000", 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.136e-06 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930056+0000", 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930061+0000", 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9289999999999997e-06 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972435+0000", 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042373347999999998 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972479+0000", 2024-09-13T09:18:11.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4597e-05 2024-09-13T09:18:11.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972532+0000", 2024-09-13T09:18:11.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3034000000000001e-05 2024-09-13T09:18:11.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.963 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.964 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19336 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 458752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.964 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:18:11.964 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971414189000001, 2024-09-13T09:18:11.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042590488000000003, 2024-09-13T09:18:11.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19336 2024-09-13T09:18:11.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930065+0000", 2024-09-13T09:18:11.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930067+0000", 2024-09-13T09:18:11.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2110000000000001e-06 2024-09-13T09:18:11.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930074+0000", 2024-09-13T09:18:11.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4640000000000003e-06 2024-09-13T09:18:11.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930079+0000", 2024-09-13T09:18:11.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7709999999999997e-06 2024-09-13T09:18:11.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972560+0000", 2024-09-13T09:18:11.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042481738999999998 2024-09-13T09:18:11.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972604+0000", 2024-09-13T09:18:11.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4060999999999997e-05 2024-09-13T09:18:11.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972656+0000", 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.164e-05 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19337 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.971397014000001, 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042697997000000001, 2024-09-13T09:18:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19337 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930082+0000", 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930085+0000", 2024-09-13T09:18:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.221e-06 2024-09-13T09:18:11.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930091+0000", 2024-09-13T09:18:11.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.942e-06 2024-09-13T09:18:11.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930096+0000", 2024-09-13T09:18:11.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.04e-06 2024-09-13T09:18:11.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972683+0000", 2024-09-13T09:18:11.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042587885999999998 2024-09-13T09:18:11.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972728+0000", 2024-09-13T09:18:11.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4058999999999999e-05 2024-09-13T09:18:11.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972781+0000", 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3156999999999999e-05 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19342 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1048576~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.969479587, 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041418260999999998, 2024-09-13T09:18:11.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19342 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932002+0000", 2024-09-13T09:18:11.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3180000000000001e-06 2024-09-13T09:18:11.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932008+0000", 2024-09-13T09:18:11.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2199999999999997e-06 2024-09-13T09:18:11.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932013+0000", 2024-09-13T09:18:11.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:18:11.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973319+0000", 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041305738000000002 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973364+0000", 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5809000000000001e-05 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973418+0000", 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3888000000000003e-05 2024-09-13T09:18:11.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19343 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1146880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.96945238, 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041515107000000002, 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.979 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.980 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19343 2024-09-13T09:18:11.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.980 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:11.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:11.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932034+0000", 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932040+0000", 2024-09-13T09:18:11.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2260000000000002e-06 2024-09-13T09:18:11.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932045+0000", 2024-09-13T09:18:11.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5589999999999999e-06 2024-09-13T09:18:11.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973446+0000", 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041401070999999998 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973491+0000", 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5021999999999997e-05 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973542+0000", 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1465000000000003e-05 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.983 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.984 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19344 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.984 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:18:11.984 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.969429853999999, 2024-09-13T09:18:11.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041617848999999998, 2024-09-13T09:18:11.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19344 2024-09-13T09:18:11.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.985 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:18:11.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:18:11.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932049+0000", 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932052+0000", 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4540000000000001e-06 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932064+0000", 2024-09-13T09:18:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1911e-05 2024-09-13T09:18:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932079+0000", 2024-09-13T09:18:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4765e-05 2024-09-13T09:18:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973570+0000", 2024-09-13T09:18:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041491800000000002 2024-09-13T09:18:11.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973614+0000", 2024-09-13T09:18:11.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4149e-05 2024-09-13T09:18:11.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973668+0000", 2024-09-13T09:18:11.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3198999999999997e-05 2024-09-13T09:18:11.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.988 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.989 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.989 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19345 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1343488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.989 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:18:11.989 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.969393683, 2024-09-13T09:18:11.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041707699000000001, 2024-09-13T09:18:11.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19345 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932085+0000", 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932089+0000", 2024-09-13T09:18:11.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7009999999999998e-06 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932097+0000", 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3010000000000007e-06 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932104+0000", 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9260000000000003e-06 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973695+0000", 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041591056000000001 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973741+0000", 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6003e-05 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973794+0000", 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2265999999999997e-05 2024-09-13T09:18:11.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19346 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1441792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.969369582000001, 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041808933999999999, 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.994 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19346 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:11.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:11.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:11.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:11.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932112+0000", 2024-09-13T09:18:11.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:18:11.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:11.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932119+0000", 2024-09-13T09:18:11.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8369999999999999e-06 2024-09-13T09:18:11.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:11.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932125+0000", 2024-09-13T09:18:11.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:11.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:11.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973822+0000", 2024-09-13T09:18:11.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041697369999999997 2024-09-13T09:18:11.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973867+0000", 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4648999999999999e-05 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973919+0000", 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2451000000000003e-05 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19347 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1540096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.969350129, 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041915045999999997, 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19347 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:11.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:12.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:12.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:18:12.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:12.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:12.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:12.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:12.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:18:12.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:12.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:12.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932129+0000", 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932131+0000", 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.278e-06 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932139+0000", 2024-09-13T09:18:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.43e-06 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932145+0000", 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7939999999999999e-06 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973946+0000", 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041801724999999998 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:12.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973993+0000", 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6245000000000002e-05 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.974045+0000", 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1950999999999998e-05 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:12.003 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:12.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:12.005 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:11.998+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:12.031 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:18:12.034 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:18:12.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T09:18:12.124 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone15 from 0xe4c443c (overlap 0x9103203) to 0x3a01401 2024-09-13T09:18:12.125 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:12.123+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:12.155 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:12.156 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:12.156 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:12.156 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:12.173 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:12.218 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:12.214+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:12.267 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-13T09:18:12.294 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:12.294 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:12.294 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:12.294 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:12.294 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:12.307 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:12.307+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:12.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:12.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:18:12.409 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:12.402+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:12.482 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:12.482 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:12.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:12.482 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:12.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.482 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4388 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.482 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:12.482 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.492572847000002, 2024-09-13T09:18:12.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069349509000000004, 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4388 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:12.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985769+0000", 2024-09-13T09:18:12.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985775+0000", 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5230000000000001e-06 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985794+0000", 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8425999999999999e-05 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985809+0000", 2024-09-13T09:18:12.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.486e-05 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.054965+0000", 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069156463000000001 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055036+0000", 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0780999999999999e-05 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055120+0000", 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3486000000000007e-05 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4389 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.492522323999999, 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069473833999999998, 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4389 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985820+0000", 2024-09-13T09:18:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985859+0000", 2024-09-13T09:18:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9279999999999997e-05 2024-09-13T09:18:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985881+0000", 2024-09-13T09:18:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2116999999999998e-05 2024-09-13T09:18:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985896+0000", 2024-09-13T09:18:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4698999999999999e-05 2024-09-13T09:18:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055152+0000", 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069256568000000004 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055220+0000", 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8255000000000003e-05 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055294+0000", 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3858999999999999e-05 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4390 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.492426712, 2024-09-13T09:18:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069563635999999998, 2024-09-13T09:18:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4390 2024-09-13T09:18:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985915+0000", 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985922+0000", 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985941+0000", 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8260000000000001e-05 2024-09-13T09:18:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985955+0000", 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4409999999999999e-05 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055325+0000", 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069370290000000001 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055395+0000", 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0158000000000002e-05 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055480+0000", 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4340999999999998e-05 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.492375644999999, 2024-09-13T09:18:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:18:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:18:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:18:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:18:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:18:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:18:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:18:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:18:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:18:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:18:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:18:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.492326908999999, 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:18:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.492259239999999, 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:18:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:18:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:18:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:18:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:18:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.492119031000001, 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:18:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:18:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:18:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:18:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:18:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.492028165000001, 2024-09-13T09:18:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:18:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:18:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.491953304999999, 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:18:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.491882129, 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:18:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.491833075999999, 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.49177134, 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:18:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:18:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.49172167, 2024-09-13T09:18:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:18:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:18:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:18:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:18:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.491673151000001, 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:18:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:18:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.491624755, 2024-09-13T09:18:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:18:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:18:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:18:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:18:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:18:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:18:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:18:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:18:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.491577467999999, 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:18:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.491494066000001, 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:18:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:18:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.491420340000001, 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:18:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:18:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:18:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:18:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:18:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:18:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:18:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.491352216999999, 2024-09-13T09:18:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:18:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:18:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:18:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:18:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:18:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:18:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:18:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:18:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.491301750000002, 2024-09-13T09:18:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:18:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:12.577 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:12.577 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:12.579 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:12.579 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:12.580 INFO:teuthology.orchestra.run.smithi032.stdout:2462 write 0x1e4bd1e thru 0x1e57705 (0xb9e8 bytes) 2024-09-13T09:18:12.580 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:12.581 INFO:teuthology.orchestra.run.smithi032.stdout:2463 read 0x8d1beec thru 0x8d1d6b5 (0x17ca bytes) 2024-09-13T09:18:12.581 INFO:teuthology.orchestra.run.smithi032.stdout:2464 read 0x4b2f22b thru 0x4b397c5 (0xa59b bytes) 2024-09-13T09:18:12.581 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:12.581 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:12.581 INFO:teuthology.orchestra.run.smithi032.stdout:2468 trunc from 0xe4c443c to 0xd55bd10 2024-09-13T09:18:12.581 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:12.582 INFO:teuthology.orchestra.run.smithi032.stdout:2469 read 0xc294028 thru 0xc29ec25 (0xabfe bytes) 2024-09-13T09:18:12.582 INFO:teuthology.orchestra.run.smithi032.stdout:2470 trunc from 0xd55bd10 to 0x91931bc 2024-09-13T09:18:12.582 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:12.583 INFO:teuthology.orchestra.run.smithi032.stdout:2471 write 0x3c0b83e thru 0x3c15cfb (0xa4be bytes) 2024-09-13T09:18:12.583 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:12.575+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:12.584 INFO:teuthology.orchestra.run.smithi032.stdout:2473 punch from 0x6f17489 to 0x6f1887a, (0x13f1 bytes) 2024-09-13T09:18:12.584 INFO:teuthology.orchestra.run.smithi032.stdout:2474 trunc from 0x91931bc to 0xa0f21d0 2024-09-13T09:18:12.590 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:12.602 INFO:teuthology.orchestra.run.smithi032.stdout:2475 write 0xd23a173 thru 0xd245bfd (0xba8b bytes) 2024-09-13T09:18:12.607 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:12.607 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:12.607 INFO:teuthology.orchestra.run.smithi032.stdout:2478 write 0x457c9eb thru 0x457f499 (0x2aaf bytes) 2024-09-13T09:18:12.610 INFO:teuthology.orchestra.run.smithi032.stdout:2480 writesame 0x54c908e thru 0x54d1ed7 data_size 0x27(0x8e4a bytes) 2024-09-13T09:18:12.612 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:12.612 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:12.614 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:12.615 INFO:teuthology.orchestra.run.smithi032.stdout:2483 punch from 0x2b34d6d to 0x2b41795, (0xca28 bytes) 2024-09-13T09:18:12.615 INFO:teuthology.orchestra.run.smithi032.stdout:2484 read 0x8797010 thru 0x87a6063 (0xf054 bytes) 2024-09-13T09:18:12.615 INFO:teuthology.orchestra.run.smithi032.stdout:2485 read 0xc481bc thru 0xc55c57 (0xda9c bytes) 2024-09-13T09:18:12.626 INFO:teuthology.orchestra.run.smithi032.stdout:2487 trunc from 0xd245bfe to 0x27ca460 2024-09-13T09:18:12.636 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:12.637 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:12.648 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:12.656 INFO:teuthology.orchestra.run.smithi032.stdout:2490 write 0x3baca84 thru 0x3bb1764 (0x4ce1 bytes) 2024-09-13T09:18:12.659 INFO:teuthology.orchestra.run.smithi032.stdout:2492 read 0x1f525f5 thru 0x1f576f9 (0x5105 bytes) 2024-09-13T09:18:12.666 INFO:teuthology.orchestra.run.smithi032.stdout:2493 trunc from 0x3bb1765 to 0x9f6361e 2024-09-13T09:18:12.672 INFO:teuthology.orchestra.run.smithi032.stdout:2494 punch from 0x7eadf6b to 0x7eb6364, (0x83f9 bytes) 2024-09-13T09:18:12.678 INFO:teuthology.orchestra.run.smithi032.stdout:2496 write 0xa270f75 thru 0xa275d77 (0x4e03 bytes) 2024-09-13T09:18:12.678 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:12.679+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:12.681 INFO:teuthology.orchestra.run.smithi032.stdout:2497 read 0x3e64c7e thru 0x3e6d6ee (0x8a71 bytes) 2024-09-13T09:18:12.682 INFO:teuthology.orchestra.run.smithi032.stdout:2498 read 0x6d4965a thru 0x6d57763 (0xe10a bytes) 2024-09-13T09:18:12.682 INFO:teuthology.orchestra.run.smithi032.stdout:2499 read 0x528734d thru 0x529717c (0xfe30 bytes) 2024-09-13T09:18:12.682 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:12.682 INFO:teuthology.orchestra.run.smithi032.stdout:2501 read 0x9426862 thru 0x942d41c (0x6bbb bytes) 2024-09-13T09:18:12.682 INFO:teuthology.orchestra.run.smithi032.stdout:2502 read 0x232762b thru 0x232fc6c (0x8642 bytes) 2024-09-13T09:18:12.682 INFO:teuthology.orchestra.run.smithi032.stdout:2504 read 0x3bdd32a thru 0x3be4e41 (0x7b18 bytes) 2024-09-13T09:18:12.682 INFO:teuthology.orchestra.run.smithi032.stdout:2505 writesame 0x7ea0016 thru 0x7ea3785 data_size 0x8(0x3770 bytes) 2024-09-13T09:18:12.684 INFO:teuthology.orchestra.run.smithi032.stdout:2507 write 0x4152844 thru 0x4159f25 (0x76e2 bytes) 2024-09-13T09:18:12.697 INFO:teuthology.orchestra.run.smithi032.stdout:2509 trunc from 0xa275d78 to 0x24deda3 2024-09-13T09:18:12.719 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:12.720 INFO:teuthology.orchestra.run.smithi032.stdout:2512 punch from 0x82a7b0 to 0x8314cb, (0x6d1b bytes) 2024-09-13T09:18:12.720 INFO:teuthology.orchestra.run.smithi032.stdout:2513 read 0x1bd49fa thru 0x1be1b37 (0xd13e bytes) 2024-09-13T09:18:12.732 INFO:teuthology.orchestra.run.smithi032.stdout:2514 write 0x54c2ebf thru 0x54c944e (0x6590 bytes) 2024-09-13T09:18:12.754 INFO:teuthology.orchestra.run.smithi032.stdout:2515 writesame 0xe2dd615 thru 0xe2e1e8a data_size 0x46(0x4876 bytes) 2024-09-13T09:18:12.755 INFO:teuthology.orchestra.run.smithi032.stdout:2519 write 0x9072d08 thru 0x907f278 (0xc571 bytes) 2024-09-13T09:18:12.759 INFO:teuthology.orchestra.run.smithi032.stdout:2520 read 0x5386cfe thru 0x5394922 (0xdc25 bytes) 2024-09-13T09:18:12.760 INFO:teuthology.orchestra.run.smithi032.stdout:2521 punch from 0x649874 to 0x6573e4, (0xdb70 bytes) 2024-09-13T09:18:12.760 INFO:teuthology.orchestra.run.smithi032.stdout:2523 punch from 0x8446c58 to 0x845614f, (0xf4f7 bytes) 2024-09-13T09:18:12.760 INFO:teuthology.orchestra.run.smithi032.stdout:2524 writesame 0x26c3fb7 thru 0x26cec2d data_size 0x1(0xac77 bytes) 2024-09-13T09:18:12.762 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:12.762 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:12.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:12.779 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:12.774+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:12.808 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:12.809 INFO:teuthology.orchestra.run.smithi032.stdout:2526 read 0x355d120 thru 0x3564880 (0x7761 bytes) 2024-09-13T09:18:12.809 INFO:teuthology.orchestra.run.smithi032.stdout:2527 trunc from 0xe2e1e8b to 0xe92f6b2 2024-09-13T09:18:12.820 INFO:teuthology.orchestra.run.smithi032.stdout:2528 punch from 0x548cc8f to 0x5490bbe, (0x3f2f bytes) 2024-09-13T09:18:12.821 INFO:teuthology.orchestra.run.smithi032.stdout:2530 writesame 0x680cb06 thru 0x6818464 data_size 0x1(0xb95f bytes) 2024-09-13T09:18:12.881 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:12.879+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:12.884 INFO:teuthology.orchestra.run.smithi032.stdout:2532 write 0x9e39417 thru 0x9e47c67 (0xe851 bytes) 2024-09-13T09:18:12.893 INFO:teuthology.orchestra.run.smithi032.stdout:2533 write 0x6a6538d thru 0x6a6e133 (0x8da7 bytes) 2024-09-13T09:18:12.899 INFO:teuthology.orchestra.run.smithi032.stdout:2534 punch from 0xacb68a8 to 0xacc2d69, (0xc4c1 bytes) 2024-09-13T09:18:12.899 INFO:teuthology.orchestra.run.smithi032.stdout:2536 read 0xdc8761a thru 0xdc894d3 (0x1eba bytes) 2024-09-13T09:18:12.899 INFO:teuthology.orchestra.run.smithi032.stdout:2537 write 0xee0acc thru 0xee9fde (0x9513 bytes) 2024-09-13T09:18:12.900 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:12.900 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:12.908 INFO:teuthology.orchestra.run.smithi032.stdout:2539 writesame 0x55132d4 thru 0x551f495 data_size 0x15(0xc1c2 bytes) 2024-09-13T09:18:12.914 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:12.923 INFO:teuthology.orchestra.run.smithi032.stdout:2540 trunc from 0xe92f6b2 to 0xbc77de5 2024-09-13T09:18:12.948 INFO:teuthology.orchestra.run.smithi032.stdout:2542 trunc from 0xbc77de5 to 0x739ca70 2024-09-13T09:18:12.981 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:12.978+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:12.997 INFO:teuthology.orchestra.run.smithi032.stdout:2543 write 0xc773d7f thru 0xc77e8b0 (0xab32 bytes) 2024-09-13T09:18:13.004 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.012 INFO:teuthology.orchestra.run.smithi032.stdout:2545 trunc from 0xc77e8b1 to 0x9abe44b 2024-09-13T09:18:13.037 INFO:teuthology.orchestra.run.smithi032.stdout:2546 writesame 0x3dc3399 thru 0x3dc9f2e data_size 0x2f(0x6b96 bytes) 2024-09-13T09:18:13.045 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T09:18:13.046 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:13.046 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:13.048 INFO:teuthology.orchestra.run.smithi032.stdout:2547 write 0x2759355 thru 0x275f47f (0x612b bytes) 2024-09-13T09:18:13.051 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.052 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.052 INFO:teuthology.orchestra.run.smithi032.stdout:2551 punch from 0x16b9c02 to 0x16bc93d, (0x2d3b bytes) 2024-09-13T09:18:13.052 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:13.067 INFO:teuthology.orchestra.run.smithi032.stdout:2554 writesame 0xb459ad4 thru 0xb45ceb3 data_size 0xa(0x33e0 bytes) 2024-09-13T09:18:13.067 INFO:teuthology.orchestra.run.smithi032.stdout:2555 writesame 0x25d3738 thru 0x25da175 data_size 0x9(0x6a3e bytes) 2024-09-13T09:18:13.070 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T09:18:13.071 INFO:teuthology.orchestra.run.smithi032.stdout:2556 write 0x99ea912 thru 0x99f7683 (0xcd72 bytes) 2024-09-13T09:18:13.092 INFO:teuthology.orchestra.run.smithi032.stdout:2559 trunc from 0xb45ceb4 to 0x3945e5b 2024-09-13T09:18:13.101 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:13.098+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:13.107 INFO:teuthology.orchestra.run.smithi032.stdout:2560 trunc from 0x3945e5b to 0x4c6021f 2024-09-13T09:18:13.114 INFO:teuthology.orchestra.run.smithi032.stdout:2561 punch from 0x38af60e to 0x38bbebc, (0xc8ae bytes) 2024-09-13T09:18:13.114 INFO:teuthology.orchestra.run.smithi032.stdout:2562 punch from 0x4b56486 to 0x4b641ab, (0xdd25 bytes) 2024-09-13T09:18:13.114 INFO:teuthology.orchestra.run.smithi032.stdout:2563 read 0x2c3d0f8 thru 0x2c4a811 (0xd71a bytes) 2024-09-13T09:18:13.114 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.124 INFO:teuthology.orchestra.run.smithi032.stdout:2567 write 0x7250ded thru 0x7254163 (0x3377 bytes) 2024-09-13T09:18:13.128 INFO:teuthology.orchestra.run.smithi032.stdout:2568 read 0x5a7acba thru 0x5a7b5c4 (0x90b bytes) 2024-09-13T09:18:13.128 INFO:teuthology.orchestra.run.smithi032.stdout:2571 punch from 0x449d7bd to 0x449f836, (0x2079 bytes) 2024-09-13T09:18:13.142 INFO:teuthology.orchestra.run.smithi032.stdout:2573 write 0xb92ae1d thru 0xb939c15 (0xedf9 bytes) 2024-09-13T09:18:13.152 INFO:teuthology.orchestra.run.smithi032.stdout:2574 trunc from 0xb939c16 to 0x744e5db 2024-09-13T09:18:13.173 INFO:teuthology.orchestra.run.smithi032.stdout:2577 trunc from 0x744e5db to 0x1905a75 2024-09-13T09:18:13.183 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:13.183+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:13.207 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.207 INFO:teuthology.orchestra.run.smithi032.stdout:2579 read 0x109c03 thru 0x117cdf (0xe0dd bytes) 2024-09-13T09:18:13.207 INFO:teuthology.orchestra.run.smithi032.stdout:2581 read 0x7e6051 thru 0x7ef301 (0x92b1 bytes) 2024-09-13T09:18:13.218 INFO:teuthology.orchestra.run.smithi032.stdout:2582 writesame 0x2e2f55e thru 0x2e3e3d6 data_size 0x1(0xee79 bytes) 2024-09-13T09:18:13.231 INFO:teuthology.orchestra.run.smithi032.stdout:2583 writesame 0x42ca0ff thru 0x42d7f0e data_size 0xbb(0xde10 bytes) 2024-09-13T09:18:13.236 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:13.236 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:13.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:13.252 INFO:teuthology.orchestra.run.smithi032.stdout:2584 writesame 0xd1ade24 thru 0xd1bc0b6 data_size 0x1(0xe293 bytes) 2024-09-13T09:18:13.271 INFO:teuthology.orchestra.run.smithi032.stdout:2585 trunc from 0xd1bc0b7 to 0x11b2937 2024-09-13T09:18:13.283 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:13.283+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:13.298 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.312 INFO:teuthology.orchestra.run.smithi032.stdout:2588 write 0x63b13d4 thru 0x63b52ef (0x3f1c bytes) 2024-09-13T09:18:13.316 INFO:teuthology.orchestra.run.smithi032.stdout:2589 punch from 0x3181edd to 0x318987b, (0x799e bytes) 2024-09-13T09:18:13.328 INFO:teuthology.orchestra.run.smithi032.stdout:2590 writesame 0x99d444b thru 0x99e2663 data_size 0x1(0xe219 bytes) 2024-09-13T09:18:13.330 INFO:teuthology.orchestra.run.smithi032.stdout:2591 trunc from 0x99e2664 to 0xb0961e1 2024-09-13T09:18:13.336 INFO:teuthology.orchestra.run.smithi032.stdout:2592 write 0x8743014 thru 0x875277c (0xf769 bytes) 2024-09-13T09:18:13.340 INFO:teuthology.orchestra.run.smithi032.stdout:2593 read 0x63ae17 thru 0x63b0de (0x2c8 bytes) 2024-09-13T09:18:13.341 INFO:teuthology.orchestra.run.smithi032.stdout:2594 punch from 0x7204dd to 0x72450b, (0x402e bytes) 2024-09-13T09:18:13.341 INFO:teuthology.orchestra.run.smithi032.stdout:2595 read 0x545a385 thru 0x546400c (0x9c88 bytes) 2024-09-13T09:18:13.341 INFO:teuthology.orchestra.run.smithi032.stdout:2596 punch from 0x20b4da9 to 0x20b94c9, (0x4720 bytes) 2024-09-13T09:18:13.341 INFO:teuthology.orchestra.run.smithi032.stdout:2597 read 0x354511f thru 0x3548e9c (0x3d7e bytes) 2024-09-13T09:18:13.341 INFO:teuthology.orchestra.run.smithi032.stdout:2598 write 0x1963af7 thru 0x1967ffc (0x4506 bytes) 2024-09-13T09:18:13.344 INFO:teuthology.orchestra.run.smithi032.stdout:2599 writesame 0x5d4fc39 thru 0x5d54e0a data_size 0x3(0x51d2 bytes) 2024-09-13T09:18:13.352 INFO:teuthology.orchestra.run.smithi032.stdout:2600 punch from 0x622dd4f to 0x623b756, (0xda07 bytes) 2024-09-13T09:18:13.352 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.352 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.352 INFO:teuthology.orchestra.run.smithi032.stdout:2603 read 0x428dc0f thru 0x4292484 (0x4876 bytes) 2024-09-13T09:18:13.352 INFO:teuthology.orchestra.run.smithi032.stdout:2605 write 0xa5004ab thru 0xa503690 (0x31e6 bytes) 2024-09-13T09:18:13.365 INFO:teuthology.orchestra.run.smithi032.stdout:2606 write 0xbdc3692 thru 0xbdd05f9 (0xcf68 bytes) 2024-09-13T09:18:13.369 INFO:teuthology.orchestra.run.smithi032.stdout:2607 punch from 0x32b16b1 to 0x32b1e27, (0x776 bytes) 2024-09-13T09:18:13.369 INFO:teuthology.orchestra.run.smithi032.stdout:2608 punch from 0x44544d2 to 0x44623cf, (0xdefd bytes) 2024-09-13T09:18:13.369 INFO:teuthology.orchestra.run.smithi032.stdout:2609 read 0x83c56d thru 0x83cb85 (0x619 bytes) 2024-09-13T09:18:13.369 INFO:teuthology.orchestra.run.smithi032.stdout:2610 write 0xbcce75c thru 0xbcd3bd5 (0x547a bytes) 2024-09-13T09:18:13.373 INFO:teuthology.orchestra.run.smithi032.stdout:2611 read 0x9301c80 thru 0x930fe45 (0xe1c6 bytes) 2024-09-13T09:18:13.375 INFO:teuthology.orchestra.run.smithi032.stdout:2614 trunc from 0xbdd05fa to 0xa80f755 2024-09-13T09:18:13.384 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:13.383+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:13.388 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:13.388 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:13.390 INFO:teuthology.orchestra.run.smithi032.stdout:2615 writesame 0x39eae86 thru 0x39ecea4 data_size 0x1(0x201f bytes) 2024-09-13T09:18:13.397 INFO:teuthology.orchestra.run.smithi032.stdout:2616 trunc from 0xa80f755 to 0xa3662d8 2024-09-13T09:18:13.403 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:13.408 INFO:teuthology.orchestra.run.smithi032.stdout:2617 writesame 0x618995 thru 0x624ca0 data_size 0xc(0xc30c bytes) 2024-09-13T09:18:13.424 INFO:teuthology.orchestra.run.smithi032.stdout:2619 trunc from 0xa3662d8 to 0x7414926 2024-09-13T09:18:13.441 INFO:teuthology.orchestra.run.smithi032.stdout:2620 read 0x1a16e27 thru 0x1a1a0ac (0x3286 bytes) 2024-09-13T09:18:13.441 INFO:teuthology.orchestra.run.smithi032.stdout:2621 punch from 0x2cfe055 to 0x2d04b6c, (0x6b17 bytes) 2024-09-13T09:18:13.442 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.442 INFO:teuthology.orchestra.run.smithi032.stdout:2623 read 0x1686a4f thru 0x169697a (0xff2c bytes) 2024-09-13T09:18:13.442 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.442 INFO:teuthology.orchestra.run.smithi032.stdout:2625 read 0x2ea127a thru 0x2eaac89 (0x9a10 bytes) 2024-09-13T09:18:13.442 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.453 INFO:teuthology.orchestra.run.smithi032.stdout:2627 trunc from 0x7414926 to 0x1fdbb9d 2024-09-13T09:18:13.486 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:13.483+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:13.500 INFO:teuthology.orchestra.run.smithi032.stdout:2629 writesame 0xcbbf5a2 thru 0xcbc6176 data_size 0x1(0x6bd5 bytes) 2024-09-13T09:18:13.509 INFO:teuthology.orchestra.run.smithi032.stdout:2630 write 0xccf7cd7 thru 0xcd03506 (0xb830 bytes) 2024-09-13T09:18:13.516 INFO:teuthology.orchestra.run.smithi032.stdout:2631 writesame 0x3fd9f5 thru 0x40d2b2 data_size 0x3(0xf8be bytes) 2024-09-13T09:18:13.533 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:13.533 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:13.537 INFO:teuthology.orchestra.run.smithi032.stdout:2632 writesame 0x1171e4b thru 0x117ba41 data_size 0x1(0x9bf7 bytes) 2024-09-13T09:18:13.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:13.579 INFO:teuthology.orchestra.run.smithi032.stdout:2633 writesame 0x93b5953 thru 0x93c3417 data_size 0x17(0xdac5 bytes) 2024-09-13T09:18:13.591 INFO:teuthology.orchestra.run.smithi032.stdout:2634 read 0x79820a0 thru 0x7985c2b (0x3b8c bytes) 2024-09-13T09:18:13.591 INFO:teuthology.orchestra.run.smithi032.stdout:2636 read 0x7eb143b thru 0x7eba6fa (0x92c0 bytes) 2024-09-13T09:18:13.592 INFO:teuthology.orchestra.run.smithi032.stdout:2638 write 0x26d9acb thru 0x26ddbba (0x40f0 bytes) 2024-09-13T09:18:13.593 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:13.590+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:13.597 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.597 INFO:teuthology.orchestra.run.smithi032.stdout:2641 read 0x56ca5b thru 0x57a52d (0xdad3 bytes) 2024-09-13T09:18:13.598 INFO:teuthology.orchestra.run.smithi032.stdout:2642 write 0x6cfdbaf thru 0x6d006d3 (0x2b25 bytes) 2024-09-13T09:18:13.607 INFO:teuthology.orchestra.run.smithi032.stdout:2643 write 0xcb432bb thru 0xcb51671 (0xe3b7 bytes) 2024-09-13T09:18:13.615 INFO:teuthology.orchestra.run.smithi032.stdout:2644 write 0x82499a4 thru 0x82541cc (0xa829 bytes) 2024-09-13T09:18:13.623 INFO:teuthology.orchestra.run.smithi032.stdout:2645 trunc from 0xcd03507 to 0xca7186f 2024-09-13T09:18:13.639 INFO:teuthology.orchestra.run.smithi032.stdout:2646 read 0x12a08b3 thru 0x12a638f (0x5add bytes) 2024-09-13T09:18:13.640 INFO:teuthology.orchestra.run.smithi032.stdout:2647 writesame 0x8d85cdc thru 0x8d8fdc1 data_size 0xf(0xa0e6 bytes) 2024-09-13T09:18:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:2648 punch from 0xa466553 to 0xa46efc4, (0x8a71 bytes) 2024-09-13T09:18:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:2649 write 0x7a4d290 thru 0x7a5bca5 (0xea16 bytes) 2024-09-13T09:18:13.653 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.665 INFO:teuthology.orchestra.run.smithi032.stdout:2651 trunc from 0xca7186f to 0x841d75e 2024-09-13T09:18:13.678 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:13.678 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:13.687 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:13.682+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:13.693 INFO:teuthology.orchestra.run.smithi032.stdout:2652 punch from 0xde8b1b to 0xdeeb0a, (0x5fef bytes) 2024-09-13T09:18:13.693 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.693 INFO:teuthology.orchestra.run.smithi032.stdout:2655 read 0x3dc0bd3 thru 0x3dcbb6d (0xaf9b bytes) 2024-09-13T09:18:13.693 INFO:teuthology.orchestra.run.smithi032.stdout:2659 write 0x496a8d4 thru 0x4972b75 (0x82a2 bytes) 2024-09-13T09:18:13.694 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:13.700 INFO:teuthology.orchestra.run.smithi032.stdout:2660 punch from 0x3dd823b to 0x3de6518, (0xe2dd bytes) 2024-09-13T09:18:13.702 INFO:teuthology.orchestra.run.smithi032.stdout:2662 trunc from 0x841d75e to 0x8aea233 2024-09-13T09:18:13.714 INFO:teuthology.orchestra.run.smithi032.stdout:2663 write 0x391ae7a thru 0x39275a4 (0xc72b bytes) 2024-09-13T09:18:13.724 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:13.724 INFO:teuthology.orchestra.run.smithi032.stdout:2665 read 0x4af61ed thru 0x4afafb8 (0x4dcc bytes) 2024-09-13T09:18:13.724 INFO:teuthology.orchestra.run.smithi032.stdout:2666 read 0x165b4a thru 0x174efa (0xf3b1 bytes) 2024-09-13T09:18:13.725 INFO:teuthology.orchestra.run.smithi032.stdout:2669 write 0x2aa75f6 thru 0x2aad235 (0x5c40 bytes) 2024-09-13T09:18:13.732 INFO:teuthology.orchestra.run.smithi032.stdout:2670 read 0x5afb2d0 thru 0x5b04895 (0x95c6 bytes) 2024-09-13T09:18:13.732 INFO:teuthology.orchestra.run.smithi032.stdout:2671 read 0x33a2dbd thru 0x33af39d (0xc5e1 bytes) 2024-09-13T09:18:13.754 INFO:teuthology.orchestra.run.smithi032.stdout:2673 write 0xc447990 thru 0xc44f49b (0x7b0c bytes) 2024-09-13T09:18:13.761 INFO:teuthology.orchestra.run.smithi032.stdout:2674 punch from 0x4a30810 to 0x4a3d94f, (0xd13f bytes) 2024-09-13T09:18:13.761 INFO:teuthology.orchestra.run.smithi032.stdout:2675 write 0xbcabcd8 thru 0xbcb1b51 (0x5e7a bytes) 2024-09-13T09:18:13.768 INFO:teuthology.orchestra.run.smithi032.stdout:2676 punch from 0x5efa542 to 0x5efdf85, (0x3a43 bytes) 2024-09-13T09:18:13.768 INFO:teuthology.orchestra.run.smithi032.stdout:2677 write 0x4e900d2 thru 0x4e95d01 (0x5c30 bytes) 2024-09-13T09:18:13.775 INFO:teuthology.orchestra.run.smithi032.stdout:2679 read 0x475e1f9 thru 0x476cae6 (0xe8ee bytes) 2024-09-13T09:18:13.775 INFO:teuthology.orchestra.run.smithi032.stdout:2680 writesame 0x46ebb8b thru 0x46ee58c data_size 0x13(0x2a02 bytes) 2024-09-13T09:18:13.779 INFO:teuthology.orchestra.run.smithi032.stdout:2682 read 0x1f8615 thru 0x1fd13a (0x4b26 bytes) 2024-09-13T09:18:13.779 INFO:teuthology.orchestra.run.smithi032.stdout:2685 read 0x6cb16e9 thru 0x6cb1edf (0x7f7 bytes) 2024-09-13T09:18:13.779 INFO:teuthology.orchestra.run.smithi032.stdout:2687 write 0x80883a1 thru 0x8088693 (0x2f3 bytes) 2024-09-13T09:18:13.782 INFO:teuthology.orchestra.run.smithi032.stdout:2688 clone 17 order 19 su 131072 sc 7 2024-09-13T09:18:13.787 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:13.787+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:13.864 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:13.864 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:13.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:13.889 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:13.887+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:13.990 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:13.987+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:14.005 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:14.005 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:14.020 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T09:18:14.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:18:14.074 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:18:14.118 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:14.119+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:14.161 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone16 from 0xc44f49c (overlap 0x11b2937) to 0x1c50eb 2024-09-13T09:18:14.171 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:14.171 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:14.192 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:14.191+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:14.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:14.292 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:14.291+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:14.302 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-13T09:18:14.384 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:14.384 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:14.393 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:14.391+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:14.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:14.430 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:14.431 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:14.431 INFO:teuthology.orchestra.run.smithi032.stdout:2691 write 0x749490a thru 0x74966ce (0x1dc5 bytes) 2024-09-13T09:18:14.459 INFO:teuthology.orchestra.run.smithi032.stdout:2692 trunc from 0xc44f49c to 0x243768e 2024-09-13T09:18:14.495 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:14.490+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:14.521 INFO:teuthology.orchestra.run.smithi032.stdout:2693 write 0xd874b0b thru 0xd882d47 (0xe23d bytes) 2024-09-13T09:18:14.529 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:14.529 INFO:teuthology.orchestra.run.smithi032.stdout:2695 write 0x1396319 thru 0x139b938 (0x5620 bytes) 2024-09-13T09:18:14.537 INFO:teuthology.orchestra.run.smithi032.stdout:2696 punch from 0x205377d to 0x2060dd6, (0xd659 bytes) 2024-09-13T09:18:14.551 INFO:teuthology.orchestra.run.smithi032.stdout:2697 trunc from 0xd882d48 to 0x43a451c 2024-09-13T09:18:14.561 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:14.561 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:14.573 INFO:teuthology.orchestra.run.smithi032.stdout:2700 punch from 0x33b3c50 to 0x33c0cf9, (0xd0a9 bytes) 2024-09-13T09:18:14.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:14.580 INFO:teuthology.orchestra.run.smithi032.stdout:2701 trunc from 0x43a451c to 0x7f8886 2024-09-13T09:18:14.595 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:14.590+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:14.621 INFO:teuthology.orchestra.run.smithi032.stdout:2702 read 0x2fe1cb thru 0x307400 (0x9236 bytes) 2024-09-13T09:18:14.638 INFO:teuthology.orchestra.run.smithi032.stdout:2704 writesame 0x2e92aca thru 0x2e9bfc5 data_size 0xa(0x94fc bytes) 2024-09-13T09:18:14.652 INFO:teuthology.orchestra.run.smithi032.stdout:2706 trunc from 0x2e9bfc6 to 0xb9372e1 2024-09-13T09:18:14.656 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:14.664 INFO:teuthology.orchestra.run.smithi032.stdout:2707 writesame 0x514e6ea thru 0x5154ee1 data_size 0xc(0x67f8 bytes) 2024-09-13T09:18:14.671 INFO:teuthology.orchestra.run.smithi032.stdout:2708 read 0x71df5ec thru 0x71e9437 (0x9e4c bytes) 2024-09-13T09:18:14.671 INFO:teuthology.orchestra.run.smithi032.stdout:2709 write 0xa6f4fde thru 0xa702d62 (0xdd85 bytes) 2024-09-13T09:18:14.678 INFO:teuthology.orchestra.run.smithi032.stdout:2710 writesame 0x94f7d93 thru 0x94fb4ca data_size 0x26(0x3738 bytes) 2024-09-13T09:18:14.683 INFO:teuthology.orchestra.run.smithi032.stdout:2711 read 0x3d9f40f thru 0x3da479b (0x538d bytes) 2024-09-13T09:18:14.683 INFO:teuthology.orchestra.run.smithi032.stdout:2712 punch from 0x57f5fb2 to 0x57fec25, (0x8c73 bytes) 2024-09-13T09:18:14.693 INFO:teuthology.orchestra.run.smithi032.stdout:2713 trunc from 0xb9372e1 to 0x4d9239f 2024-09-13T09:18:14.696 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:14.695+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:14.714 INFO:teuthology.orchestra.run.smithi032.stdout:2715 punch from 0x4644e17 to 0x4652534, (0xd71d bytes) 2024-09-13T09:18:14.714 INFO:teuthology.orchestra.run.smithi032.stdout:2716 read 0x1463c70 thru 0x14733a3 (0xf734 bytes) 2024-09-13T09:18:14.715 INFO:teuthology.orchestra.run.smithi032.stdout:2717 trunc from 0x4d9239f to 0x45d5674 2024-09-13T09:18:14.720 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:14.720 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:14.727 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:14.727 INFO:teuthology.orchestra.run.smithi032.stdout:2719 read 0x383467d thru 0x3836346 (0x1cca bytes) 2024-09-13T09:18:14.729 INFO:teuthology.orchestra.run.smithi032.stdout:2721 trunc from 0x45d5674 to 0x31b9a13 2024-09-13T09:18:14.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:14.746 INFO:teuthology.orchestra.run.smithi032.stdout:2723 trunc from 0x31b9a13 to 0x61fb090 2024-09-13T09:18:14.757 INFO:teuthology.orchestra.run.smithi032.stdout:2724 write 0x2082aaa thru 0x2087814 (0x4d6b bytes) 2024-09-13T09:18:14.769 INFO:teuthology.orchestra.run.smithi032.stdout:2725 trunc from 0x61fb090 to 0x9a17cd2 2024-09-13T09:18:14.782 INFO:teuthology.orchestra.run.smithi032.stdout:2729 read 0x43ccf7d thru 0x43dbd05 (0xed89 bytes) 2024-09-13T09:18:14.782 INFO:teuthology.orchestra.run.smithi032.stdout:2730 write 0x6e47df1 thru 0x6e56d89 (0xef99 bytes) 2024-09-13T09:18:14.787 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:14.787 INFO:teuthology.orchestra.run.smithi032.stdout:2732 write 0x3595cdd thru 0x359e2a3 (0x85c7 bytes) 2024-09-13T09:18:14.797 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:14.795+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:14.805 INFO:teuthology.orchestra.run.smithi032.stdout:2733 write 0xb7a2dce thru 0xb7abda4 (0x8fd7 bytes) 2024-09-13T09:18:14.811 INFO:teuthology.orchestra.run.smithi032.stdout:2734 write 0x78d3129 thru 0x78dcd4e (0x9c26 bytes) 2024-09-13T09:18:14.816 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:14.817 INFO:teuthology.orchestra.run.smithi032.stdout:2736 read 0x4ac3abd thru 0x4ac7b91 (0x40d5 bytes) 2024-09-13T09:18:14.817 INFO:teuthology.orchestra.run.smithi032.stdout:2739 write 0x3cf5d86 thru 0x3cfa046 (0x42c1 bytes) 2024-09-13T09:18:14.829 INFO:teuthology.orchestra.run.smithi032.stdout:2741 trunc from 0xb7abda5 to 0x7198b99 2024-09-13T09:18:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:14.848 INFO:teuthology.orchestra.run.smithi032.stdout:2742 write 0x4442d8b thru 0x444a51b (0x7791 bytes) 2024-09-13T09:18:14.853 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:14.853 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:14.853 INFO:teuthology.orchestra.run.smithi032.stdout:2743 clone 18 order 21 su 32768 sc 8 2024-09-13T09:18:14.869 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:14.872 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:14.872 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:14.873 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:14.873 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:14.873 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:14.887 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:14.890 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:14.911 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:14.911 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:14.911+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:14.931 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:14.952 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:14.953 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:15.011 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:15.011 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:15.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:15.011 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:15.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.011 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5479 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.011 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:18:15.012 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.470209401999998, 2024-09-13T09:18:15.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027937460000000001, 2024-09-13T09:18:15.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5479 2024-09-13T09:18:15.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.012 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539251+0000", 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539250+0000", 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539256+0000", 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539263+0000", 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3779999999999998e-06 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539273+0000", 2024-09-13T09:18:15.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0457e-05 2024-09-13T09:18:15.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567010+0000", 2024-09-13T09:18:15.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027736878 2024-09-13T09:18:15.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567083+0000", 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3187000000000002e-05 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567188+0000", 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010491499999999999 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5480 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 413696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.016 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.470182183999999, 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028132641999999999, 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5480 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539278+0000", 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539280+0000", 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539297+0000", 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7147999999999999e-05 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539303+0000", 2024-09-13T09:18:15.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9039999999999997e-06 2024-09-13T09:18:15.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567230+0000", 2024-09-13T09:18:15.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027927250000000001 2024-09-13T09:18:15.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567316+0000", 2024-09-13T09:18:15.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5970999999999994e-05 2024-09-13T09:18:15.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567411+0000", 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4523000000000003e-05 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5481 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.470152188, 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028317978000000001, 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.021 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5481 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539308+0000", 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539307+0000", 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539311+0000", 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.264e-06 2024-09-13T09:18:15.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539317+0000", 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539322+0000", 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6949999999999999e-06 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567446+0000", 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028124341000000001 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567527+0000", 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0785000000000007e-05 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567626+0000", 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9103000000000003e-05 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.025 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5482 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.470134363, 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028619326, 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.026 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5482 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539326+0000", 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539325+0000", 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539333+0000", 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.055e-06 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539340+0000", 2024-09-13T09:18:15.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4980000000000003e-06 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539344+0000", 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5369999999999999e-06 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567655+0000", 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028310906 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567732+0000", 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6829999999999995e-05 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567945+0000", 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000212909 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5483 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.030 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:15.031 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.470111102000001, 2024-09-13T09:18:15.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028784015999999999, 2024-09-13T09:18:15.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5483 2024-09-13T09:18:15.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.031 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539349+0000", 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539351+0000", 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539357+0000", 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0090000000000004e-06 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539361+0000", 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.357e-06 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.567974+0000", 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028612385000000001 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568050+0000", 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5820000000000003e-05 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568133+0000", 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3551e-05 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5484 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.470094549999999, 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028949923999999998, 2024-09-13T09:18:15.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5484 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539366+0000", 2024-09-13T09:18:15.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539365+0000", 2024-09-13T09:18:15.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539367+0000", 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539374+0000", 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.539378+0000", 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:18:15.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568159+0000", 2024-09-13T09:18:15.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028780490999999998 2024-09-13T09:18:15.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568236+0000", 2024-09-13T09:18:15.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7156000000000005e-05 2024-09-13T09:18:15.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568316+0000", 2024-09-13T09:18:15.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9598999999999995e-05 2024-09-13T09:18:15.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.040 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5485 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.040 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:15.040 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469274671000001, 2024-09-13T09:18:15.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028332593, 2024-09-13T09:18:15.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5485 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540185+0000", 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540189+0000", 2024-09-13T09:18:15.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:18:15.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540198+0000", 2024-09-13T09:18:15.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9099999999999994e-06 2024-09-13T09:18:15.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540206+0000", 2024-09-13T09:18:15.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-13T09:18:15.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568360+0000", 2024-09-13T09:18:15.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028154324000000001 2024-09-13T09:18:15.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568436+0000", 2024-09-13T09:18:15.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5821000000000005e-05 2024-09-13T09:18:15.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568518+0000", 2024-09-13T09:18:15.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1895000000000001e-05 2024-09-13T09:18:15.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.045 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.045 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5486 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.045 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:15.045 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469221661999999, 2024-09-13T09:18:15.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028479536999999999, 2024-09-13T09:18:15.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5486 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540238+0000", 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540241+0000", 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4290000000000001e-06 2024-09-13T09:18:15.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540249+0000", 2024-09-13T09:18:15.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9480000000000008e-06 2024-09-13T09:18:15.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540255+0000", 2024-09-13T09:18:15.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6019999999999999e-06 2024-09-13T09:18:15.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568543+0000", 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028288260999999999 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568636+0000", 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2627999999999995e-05 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568718+0000", 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2213999999999996e-05 2024-09-13T09:18:15.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5487 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469200576999999, 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028653435000000001, 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5487 2024-09-13T09:18:15.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.051 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:18:15.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540260+0000", 2024-09-13T09:18:15.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540259+0000", 2024-09-13T09:18:15.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540262+0000", 2024-09-13T09:18:15.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.413e-06 2024-09-13T09:18:15.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540268+0000", 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3940000000000001e-06 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540272+0000", 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3989999999999997e-06 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568745+0000", 2024-09-13T09:18:15.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028472173999999999 2024-09-13T09:18:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568822+0000", 2024-09-13T09:18:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7694000000000005e-05 2024-09-13T09:18:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568913+0000", 2024-09-13T09:18:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0779000000000006e-05 2024-09-13T09:18:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.054 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.055 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5488 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 430080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.055 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:15.055 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469183177000001, 2024-09-13T09:18:15.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028788326999999999, 2024-09-13T09:18:15.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5488 2024-09-13T09:18:15.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.056 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:15.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540277+0000", 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540279+0000", 2024-09-13T09:18:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2409999999999998e-06 2024-09-13T09:18:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540285+0000", 2024-09-13T09:18:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3180000000000003e-06 2024-09-13T09:18:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540300+0000", 2024-09-13T09:18:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4625e-05 2024-09-13T09:18:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.568939+0000", 2024-09-13T09:18:15.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028639688 2024-09-13T09:18:15.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569016+0000", 2024-09-13T09:18:15.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6556000000000004e-05 2024-09-13T09:18:15.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569065+0000", 2024-09-13T09:18:15.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9246000000000003e-05 2024-09-13T09:18:15.060 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.060 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.060 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.060 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5489 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.060 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:18:15.060 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469155582999999, 2024-09-13T09:18:15.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028875332, 2024-09-13T09:18:15.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5489 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540305+0000", 2024-09-13T09:18:15.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540304+0000", 2024-09-13T09:18:15.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540307+0000", 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4370000000000001e-06 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540313+0000", 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540318+0000", 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569079+0000", 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028760714 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569128+0000", 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8702999999999998e-05 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569180+0000", 2024-09-13T09:18:15.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2105999999999999e-05 2024-09-13T09:18:15.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.065 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.065 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5490 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.065 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:15.065 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469137088, 2024-09-13T09:18:15.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029041027000000001, 2024-09-13T09:18:15.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.065 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5490 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:15.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540323+0000", 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540326+0000", 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9840000000000001e-06 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540332+0000", 2024-09-13T09:18:15.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9730000000000002e-06 2024-09-13T09:18:15.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540338+0000", 2024-09-13T09:18:15.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3550000000000001e-06 2024-09-13T09:18:15.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569202+0000", 2024-09-13T09:18:15.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.028863521 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569276+0000", 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569364+0000", 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7594000000000002e-05 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.070 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5491 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.070 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:15.070 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469117703999999, 2024-09-13T09:18:15.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029221864, 2024-09-13T09:18:15.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.070 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5491 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540342+0000", 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540350+0000", 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4250000000000001e-06 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540356+0000", 2024-09-13T09:18:15.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2960000000000004e-06 2024-09-13T09:18:15.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540361+0000", 2024-09-13T09:18:15.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:18:15.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569388+0000", 2024-09-13T09:18:15.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029026744 2024-09-13T09:18:15.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569474+0000", 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6519000000000003e-05 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569564+0000", 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9969999999999994e-05 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.074 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5492 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469094407, 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029401203000000001, 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5492 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540366+0000", 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540365+0000", 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540368+0000", 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540374+0000", 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9349999999999999e-06 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540378+0000", 2024-09-13T09:18:15.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.651e-06 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569591+0000", 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029212557 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569668+0000", 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569767+0000", 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9061000000000006e-05 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5493 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469077543000001, 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029599424999999999, 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.080 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.080 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.080 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5493 2024-09-13T09:18:15.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.080 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:18:15.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540383+0000", 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540382+0000", 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540384+0000", 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0430000000000002e-06 2024-09-13T09:18:15.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540390+0000", 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540395+0000", 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5990000000000004e-06 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569795+0000", 2024-09-13T09:18:15.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029400582000000001 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569880+0000", 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5210999999999994e-05 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.569982+0000", 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101584 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5494 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469061457999999, 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029770637999999999, 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5494 2024-09-13T09:18:15.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540399+0000", 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540398+0000", 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540406+0000", 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540413+0000", 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:18:15.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540418+0000", 2024-09-13T09:18:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8289999999999998e-06 2024-09-13T09:18:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570007+0000", 2024-09-13T09:18:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029588851999999999 2024-09-13T09:18:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570084+0000", 2024-09-13T09:18:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7253e-05 2024-09-13T09:18:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570169+0000", 2024-09-13T09:18:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5391999999999998e-05 2024-09-13T09:18:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5495 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 499712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469023142000001, 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029938664, 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5495 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540437+0000", 2024-09-13T09:18:15.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540440+0000", 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0319999999999999e-06 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540446+0000", 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8549999999999996e-06 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540451+0000", 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570198+0000", 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029746060000000001 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570284+0000", 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6044999999999994e-05 2024-09-13T09:18:15.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570376+0000", 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2071000000000007e-05 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5496 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469004186999999, 2024-09-13T09:18:15.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030121617999999999, 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5496 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540456+0000", 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540458+0000", 2024-09-13T09:18:15.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:18:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540464+0000", 2024-09-13T09:18:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.156e-06 2024-09-13T09:18:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540468+0000", 2024-09-13T09:18:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:18:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570404+0000", 2024-09-13T09:18:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029935632 2024-09-13T09:18:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570488+0000", 2024-09-13T09:18:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3873000000000001e-05 2024-09-13T09:18:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570578+0000", 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9757000000000001e-05 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:5497 3.19 3:9a7ba229:::rbd_data.1185192b47c4.0000000000000001:5 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.468987623, 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030305259000000001, 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5497 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540473+0000", 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540472+0000", 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540474+0000", 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0490000000000002e-06 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540480+0000", 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1970000000000001e-06 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.540491+0000", 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0964e-05 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570608+0000", 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030116105000000001 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570688+0000", 2024-09-13T09:18:15.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0143999999999997e-05 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:44.570778+0000", 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0131999999999997e-05 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.9354555930000004, 2024-09-13T09:18:15.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:18:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:18:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:18:15.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:18:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:18:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:18:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:18:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:18:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:18:15.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:18:15.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:18:15.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:18:15.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:18:15.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.109 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.109 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.109 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:15.110 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:15.011+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:15.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:15.111 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:18:15.112 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:18:15.113 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:18:15.113 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:18:15.114 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:15.110+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:15.192 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone17 from 0x7198b99 (overlap 0x7f8886) to 0x0 2024-09-13T09:18:15.234 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:15.235+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:15.298 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:15.299 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:15.299 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:15.299 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:15.316 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:15.315+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:15.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:15.353 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-13T09:18:15.360 INFO:teuthology.orchestra.run.smithi032.stdout:2744 punch from 0x5bd3650 to 0x5bdac78, (0x7628 bytes) 2024-09-13T09:18:15.379 INFO:teuthology.orchestra.run.smithi032.stdout:2746 write 0xb994602 thru 0xb994aac (0x4ab bytes) 2024-09-13T09:18:15.385 INFO:teuthology.orchestra.run.smithi032.stdout:2747 write 0x65d02c8 thru 0x65d31ec (0x2f25 bytes) 2024-09-13T09:18:15.393 INFO:teuthology.orchestra.run.smithi032.stdout:2748 read 0x86e151 thru 0x86e4ab (0x35b bytes) 2024-09-13T09:18:15.394 INFO:teuthology.orchestra.run.smithi032.stdout:2749 write 0x9f1ffcf thru 0x9f27ed5 (0x7f07 bytes) 2024-09-13T09:18:15.401 INFO:teuthology.orchestra.run.smithi032.stdout:2750 punch from 0x5b60dfe to 0x5b6492b, (0x3b2d bytes) 2024-09-13T09:18:15.401 INFO:teuthology.orchestra.run.smithi032.stdout:2751 read 0x23ec4a6 thru 0x23f05f0 (0x414b bytes) 2024-09-13T09:18:15.412 INFO:teuthology.orchestra.run.smithi032.stdout:2752 write 0xd5fb69d thru 0xd605701 (0xa065 bytes) 2024-09-13T09:18:15.417 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:15.410+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:15.419 INFO:teuthology.orchestra.run.smithi032.stdout:2756 read 0x953b364 thru 0x954158f (0x622c bytes) 2024-09-13T09:18:15.426 INFO:teuthology.orchestra.run.smithi032.stdout:2758 trunc from 0xd605702 to 0x86aa081 2024-09-13T09:18:15.444 INFO:teuthology.orchestra.run.smithi032.stdout:2759 write 0x4cae443 thru 0x4cb0cfe (0x28bc bytes) 2024-09-13T09:18:15.454 INFO:teuthology.orchestra.run.smithi032.stdout:2760 read 0x2ca0ade thru 0x2caf065 (0xe588 bytes) 2024-09-13T09:18:15.467 INFO:teuthology.orchestra.run.smithi032.stdout:2761 write 0xb05c85b thru 0xb065c37 (0x93dd bytes) 2024-09-13T09:18:15.485 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:18:15.486 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:15.486 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:15.486 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:15.486 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:15.486 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:18:15.488 INFO:teuthology.orchestra.run.smithi032.stdout:2762 write 0xca91c47 thru 0xca95380 (0x373a bytes) 2024-09-13T09:18:15.494 INFO:teuthology.orchestra.run.smithi032.stdout:2763 trunc from 0xca95381 to 0xd4aee5d 2024-09-13T09:18:15.502 INFO:teuthology.orchestra.run.smithi032.stdout:2766 read 0x3326ff thru 0x338475 (0x5d77 bytes) 2024-09-13T09:18:15.503 INFO:teuthology.orchestra.run.smithi032.stdout:2767 read 0xb2dec4a thru 0xb2dec5a (0x11 bytes) 2024-09-13T09:18:15.503 INFO:teuthology.orchestra.run.smithi032.stdout:2768 write 0x4aba61b thru 0x4ac951a (0xef00 bytes) 2024-09-13T09:18:15.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:15.509 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-09-13T09:18:15.055552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":520761303,"num_objects":565,"num_object_clones":99,"num_object_copies":1130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":565,"num_whiteouts":135,"num_read":41349,"num_read_kb":108740,"num_write":7985,"num_write_kb":88414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":6001747,"num_keys_recovered":127,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6192,"ondisk_log_size":6192,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":274680,"kb_used_data":24020,"kb_used_omap":10,"kb_used_meta":250485,"kb_avail":660328200,"statfs":{"total":676457349120,"available":676176076800,"internally_reserved":0,"allocated":24596480,"data_stored":118210705,"data_compressed":1444224,"data_compressed_allocated":7757824,"data_compressed_original":106124719,"omap_allocated":11151,"internal_metadata":256496753},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":107783787,"num_objects":97,"num_object_clones":6,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":17,"num_read":4277,"num_read_kb":7130,"num_write":966,"num_write_kb":8071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040681"},"pg_stats":[{"pgid":"3.1f","version":"62'126","reported_seq":252,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.608782+0000","last_change":"2024-09-13T09:17:40.849739+0000","last_active":"2024-09-13T09:18:13.608782+0000","last_peered":"2024-09-13T09:18:13.608782+0000","last_clean":"2024-09-13T09:18:13.608782+0000","last_became_active":"2024-09-13T09:17:40.791187+0000","last_became_peered":"2024-09-13T09:17:40.791187+0000","last_unstale":"2024-09-13T09:18:13.608782+0000","last_undegraded":"2024-09-13T09:18:13.608782+0000","last_fullsized":"2024-09-13T09:18:13.608782+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:13.280600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046778999999999999,"stat_sum":{"num_bytes":25108591,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":383,"num_read_kb":3309,"num_write":201,"num_write_kb":8251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"61'354","reported_seq":526,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.045866+0000","last_change":"2024-09-13T09:18:00.931799+0000","last_active":"2024-09-13T09:18:13.045866+0000","last_peered":"2024-09-13T09:18:13.045866+0000","last_clean":"2024-09-13T09:18:13.045866+0000","last_became_active":"2024-09-13T09:18:00.931068+0000","last_became_peered":"2024-09-13T09:18:00.931068+0000","last_unstale":"2024-09-13T09:18:13.045866+0000","last_undegraded":"2024-09-13T09:18:13.045866+0000","last_fullsized":"2024-09-13T09:18:13.045866+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:39:18.748434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382264,"stat_sum":{"num_bytes":10875609,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":1406,"num_read_kb":1836,"num_write":552,"num_write_kb":1683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":412132,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"63'162","reported_seq":854,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:18:14.082721+0000","last_change":"2024-09-13T09:17:28.692127+0000","last_active":"2024-09-13T09:18:14.082721+0000","last_peered":"2024-09-13T09:18:14.082721+0000","last_clean":"2024-09-13T09:18:14.082721+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:18:14.082721+0000","last_undegraded":"2024-09-13T09:18:14.082721+0000","last_fullsized":"2024-09-13T09:18:14.082721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:20.509991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001448682,"stat_sum":{"num_bytes":20453247,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":1889,"num_read_kb":2719,"num_write":192,"num_write_kb":8135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"63'332","reported_seq":1511,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:18:14.129970+0000","last_change":"2024-09-13T09:17:28.691986+0000","last_active":"2024-09-13T09:18:14.129970+0000","last_peered":"2024-09-13T09:18:14.129970+0000","last_clean":"2024-09-13T09:18:14.129970+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:18:14.129970+0000","last_undegraded":"2024-09-13T09:18:14.129970+0000","last_fullsized":"2024-09-13T09:18:14.129970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:05.002231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013280939999999999,"stat_sum":{"num_bytes":1839020,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":3758,"num_read_kb":3417,"num_write":407,"num_write_kb":1380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"62'337","reported_seq":1943,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.288877+0000","last_change":"2024-09-13T09:17:40.897770+0000","last_active":"2024-09-13T09:18:13.288877+0000","last_peered":"2024-09-13T09:18:13.288877+0000","last_clean":"2024-09-13T09:18:13.288877+0000","last_became_active":"2024-09-13T09:17:40.799618+0000","last_became_peered":"2024-09-13T09:17:40.799618+0000","last_unstale":"2024-09-13T09:18:13.288877+0000","last_undegraded":"2024-09-13T09:18:13.288877+0000","last_fullsized":"2024-09-13T09:18:13.288877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:00.757881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042467999999999998,"stat_sum":{"num_bytes":13942805,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":3438,"num_read_kb":6709,"num_write":475,"num_write_kb":1788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":65341,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"61'87","reported_seq":1095,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:18:14.080926+0000","last_change":"2024-09-13T09:18:04.085791+0000","last_active":"2024-09-13T09:18:14.080926+0000","last_peered":"2024-09-13T09:18:14.080926+0000","last_clean":"2024-09-13T09:18:14.080926+0000","last_became_active":"2024-09-13T09:18:03.973063+0000","last_became_peered":"2024-09-13T09:18:03.973063+0000","last_unstale":"2024-09-13T09:18:14.080926+0000","last_undegraded":"2024-09-13T09:18:14.080926+0000","last_fullsized":"2024-09-13T09:18:14.080926+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:29.911994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077588799999999997,"stat_sum":{"num_bytes":8748288,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":800,"num_read_kb":2247,"num_write":85,"num_write_kb":1211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":889520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"61'179","reported_seq":1339,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.060555+0000","last_change":"2024-09-13T09:18:06.977335+0000","last_active":"2024-09-13T09:18:13.060555+0000","last_peered":"2024-09-13T09:18:13.060555+0000","last_clean":"2024-09-13T09:18:13.060555+0000","last_became_active":"2024-09-13T09:18:06.975016+0000","last_became_peered":"2024-09-13T09:18:06.975016+0000","last_unstale":"2024-09-13T09:18:13.060555+0000","last_undegraded":"2024-09-13T09:18:13.060555+0000","last_fullsized":"2024-09-13T09:18:13.060555+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:30.489567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023026689999999998,"stat_sum":{"num_bytes":28851875,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":1423,"num_read_kb":6353,"num_write":218,"num_write_kb":870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":439303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"62'90","reported_seq":860,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.282367+0000","last_change":"2024-09-13T09:17:40.836458+0000","last_active":"2024-09-13T09:18:13.282367+0000","last_peered":"2024-09-13T09:18:13.282367+0000","last_clean":"2024-09-13T09:18:13.282367+0000","last_became_active":"2024-09-13T09:17:40.797607+0000","last_became_peered":"2024-09-13T09:17:40.797607+0000","last_unstale":"2024-09-13T09:18:13.282367+0000","last_undegraded":"2024-09-13T09:18:13.282367+0000","last_fullsized":"2024-09-13T09:18:13.282367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:09.025976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001136335,"stat_sum":{"num_bytes":27537504,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":710,"num_read_kb":2742,"num_write":89,"num_write_kb":918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":42856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"62'167","reported_seq":761,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.521929+0000","last_change":"2024-09-13T09:17:43.811230+0000","last_active":"2024-09-13T09:18:13.521929+0000","last_peered":"2024-09-13T09:18:13.521929+0000","last_clean":"2024-09-13T09:18:13.521929+0000","last_became_active":"2024-09-13T09:17:43.809533+0000","last_became_peered":"2024-09-13T09:17:43.809533+0000","last_unstale":"2024-09-13T09:18:13.521929+0000","last_undegraded":"2024-09-13T09:18:13.521929+0000","last_fullsized":"2024-09-13T09:18:13.521929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:20.389195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001841894,"stat_sum":{"num_bytes":4085620,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":1324,"num_read_kb":1622,"num_write":158,"num_write_kb":955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":16384,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"61'97","reported_seq":553,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.520314+0000","last_change":"2024-09-13T09:17:28.692331+0000","last_active":"2024-09-13T09:18:13.520314+0000","last_peered":"2024-09-13T09:18:13.520314+0000","last_clean":"2024-09-13T09:18:13.520314+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:18:13.520314+0000","last_undegraded":"2024-09-13T09:18:13.520314+0000","last_fullsized":"2024-09-13T09:18:13.520314+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:29.849557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016477340000000001,"stat_sum":{"num_bytes":24184065,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":536,"num_read_kb":2699,"num_write":109,"num_write_kb":1013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"62'87","reported_seq":879,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.180259+0000","last_change":"2024-09-13T09:17:28.684144+0000","last_active":"2024-09-13T09:18:13.180259+0000","last_peered":"2024-09-13T09:18:13.180259+0000","last_clean":"2024-09-13T09:18:13.180259+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:18:13.180259+0000","last_undegraded":"2024-09-13T09:18:13.180259+0000","last_fullsized":"2024-09-13T09:18:13.180259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:10.717628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085639099999999997,"stat_sum":{"num_bytes":12712511,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":769,"num_read_kb":3899,"num_write":85,"num_write_kb":1271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"62'306","reported_seq":453,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.458857+0000","last_change":"2024-09-13T09:17:28.681431+0000","last_active":"2024-09-13T09:18:13.458857+0000","last_peered":"2024-09-13T09:18:13.458857+0000","last_clean":"2024-09-13T09:18:13.458857+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:18:13.458857+0000","last_undegraded":"2024-09-13T09:18:13.458857+0000","last_fullsized":"2024-09-13T09:18:13.458857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:42.341412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083138199999999998,"stat_sum":{"num_bytes":18425046,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":1178,"num_read_kb":2569,"num_write":485,"num_write_kb":7302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.063080+0000","last_change":"2024-09-13T09:17:57.918052+0000","last_active":"2024-09-13T09:18:13.063080+0000","last_peered":"2024-09-13T09:18:13.063080+0000","last_clean":"2024-09-13T09:18:13.063080+0000","last_became_active":"2024-09-13T09:17:57.917112+0000","last_became_peered":"2024-09-13T09:17:57.917112+0000","last_unstale":"2024-09-13T09:18:13.063080+0000","last_undegraded":"2024-09-13T09:18:13.063080+0000","last_fullsized":"2024-09-13T09:18:13.063080+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:44.462499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"62'128","reported_seq":170,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.370781+0000","last_change":"2024-09-13T09:18:04.241645+0000","last_active":"2024-09-13T09:18:13.370781+0000","last_peered":"2024-09-13T09:18:13.370781+0000","last_clean":"2024-09-13T09:18:13.370781+0000","last_became_active":"2024-09-13T09:18:03.979234+0000","last_became_peered":"2024-09-13T09:18:03.979234+0000","last_unstale":"2024-09-13T09:18:13.370781+0000","last_undegraded":"2024-09-13T09:18:13.370781+0000","last_fullsized":"2024-09-13T09:18:13.370781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:42:02.026772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002893765,"stat_sum":{"num_bytes":10672675,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":1183,"num_read_kb":1710,"num_write":126,"num_write_kb":930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":380627,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.060734+0000","last_change":"2024-09-13T09:18:03.969020+0000","last_active":"2024-09-13T09:18:13.060734+0000","last_peered":"2024-09-13T09:18:13.060734+0000","last_clean":"2024-09-13T09:18:13.060734+0000","last_became_active":"2024-09-13T09:18:03.968249+0000","last_became_peered":"2024-09-13T09:18:03.968249+0000","last_unstale":"2024-09-13T09:18:13.060734+0000","last_undegraded":"2024-09-13T09:18:13.060734+0000","last_fullsized":"2024-09-13T09:18:13.060734+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:00.854437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"62'109","reported_seq":888,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.674553+0000","last_change":"2024-09-13T09:17:28.684573+0000","last_active":"2024-09-13T09:18:13.674553+0000","last_peered":"2024-09-13T09:18:13.674553+0000","last_clean":"2024-09-13T09:18:13.674553+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:18:13.674553+0000","last_undegraded":"2024-09-13T09:18:13.674553+0000","last_fullsized":"2024-09-13T09:18:13.674553+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:51.225828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086084099999999999,"stat_sum":{"num_bytes":11421310,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":752,"num_read_kb":2373,"num_write":102,"num_write_kb":942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.062976+0000","last_change":"2024-09-13T09:17:40.794662+0000","last_active":"2024-09-13T09:18:13.062976+0000","last_peered":"2024-09-13T09:18:13.062976+0000","last_clean":"2024-09-13T09:18:13.062976+0000","last_became_active":"2024-09-13T09:17:40.793882+0000","last_became_peered":"2024-09-13T09:17:40.793882+0000","last_unstale":"2024-09-13T09:18:13.062976+0000","last_undegraded":"2024-09-13T09:18:13.062976+0000","last_fullsized":"2024-09-13T09:18:13.062976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:22.922753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"62'108","reported_seq":126,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.733361+0000","last_change":"2024-09-13T09:18:04.163142+0000","last_active":"2024-09-13T09:18:13.733361+0000","last_peered":"2024-09-13T09:18:13.733361+0000","last_clean":"2024-09-13T09:18:13.733361+0000","last_became_active":"2024-09-13T09:18:03.973314+0000","last_became_peered":"2024-09-13T09:18:03.973314+0000","last_unstale":"2024-09-13T09:18:13.733361+0000","last_undegraded":"2024-09-13T09:18:13.733361+0000","last_fullsized":"2024-09-13T09:18:13.733361+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.256457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024125790000000002,"stat_sum":{"num_bytes":22802988,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":241,"num_read_kb":1053,"num_write":98,"num_write_kb":1053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":167476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.060629+0000","last_change":"2024-09-13T09:18:03.968679+0000","last_active":"2024-09-13T09:18:13.060629+0000","last_peered":"2024-09-13T09:18:13.060629+0000","last_clean":"2024-09-13T09:18:13.060629+0000","last_became_active":"2024-09-13T09:18:03.967133+0000","last_became_peered":"2024-09-13T09:18:03.967133+0000","last_unstale":"2024-09-13T09:18:13.060629+0000","last_undegraded":"2024-09-13T09:18:13.060629+0000","last_fullsized":"2024-09-13T09:18:13.060629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:06.643934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"61'96","reported_seq":352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.520213+0000","last_change":"2024-09-13T09:17:28.692726+0000","last_active":"2024-09-13T09:18:13.520213+0000","last_peered":"2024-09-13T09:18:13.520213+0000","last_clean":"2024-09-13T09:18:13.520213+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:18:13.520213+0000","last_undegraded":"2024-09-13T09:18:13.520213+0000","last_fullsized":"2024-09-13T09:18:13.520213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:00.161632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018814960000000001,"stat_sum":{"num_bytes":36148301,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":283,"num_read_kb":3791,"num_write":94,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"62'74","reported_seq":1276,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.782662+0000","last_change":"2024-09-13T09:17:40.875161+0000","last_active":"2024-09-13T09:18:13.782662+0000","last_peered":"2024-09-13T09:18:13.782662+0000","last_clean":"2024-09-13T09:18:13.782662+0000","last_became_active":"2024-09-13T09:17:40.800027+0000","last_became_peered":"2024-09-13T09:17:40.800027+0000","last_unstale":"2024-09-13T09:18:13.782662+0000","last_undegraded":"2024-09-13T09:18:13.782662+0000","last_fullsized":"2024-09-13T09:18:13.782662+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:10.182783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066593499999999999,"stat_sum":{"num_bytes":11912633,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":1146,"num_read_kb":4110,"num_write":69,"num_write_kb":404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":29248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":95,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.053313+0000","last_change":"2024-09-13T09:18:03.976890+0000","last_active":"2024-09-13T09:18:13.053313+0000","last_peered":"2024-09-13T09:18:13.053313+0000","last_clean":"2024-09-13T09:18:13.053313+0000","last_became_active":"2024-09-13T09:18:03.970835+0000","last_became_peered":"2024-09-13T09:18:03.970835+0000","last_unstale":"2024-09-13T09:18:13.053313+0000","last_undegraded":"2024-09-13T09:18:13.053313+0000","last_fullsized":"2024-09-13T09:18:13.053313+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:05.189083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"62'181","reported_seq":703,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.674525+0000","last_change":"2024-09-13T09:18:04.075705+0000","last_active":"2024-09-13T09:18:13.674525+0000","last_peered":"2024-09-13T09:18:13.674525+0000","last_clean":"2024-09-13T09:18:13.674525+0000","last_became_active":"2024-09-13T09:18:03.970479+0000","last_became_peered":"2024-09-13T09:18:03.970479+0000","last_unstale":"2024-09-13T09:18:13.674525+0000","last_undegraded":"2024-09-13T09:18:13.674525+0000","last_fullsized":"2024-09-13T09:18:13.674525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:06.229840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043683300000000002,"stat_sum":{"num_bytes":8178701,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":2517,"num_read_kb":4281,"num_write":181,"num_write_kb":831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1031390,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":56,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:18:12.043048+0000","last_change":"2024-09-13T09:17:19.575553+0000","last_active":"2024-09-13T09:18:12.043048+0000","last_peered":"2024-09-13T09:18:12.043048+0000","last_clean":"2024-09-13T09:18:12.043048+0000","last_became_active":"2024-09-13T09:17:17.493930+0000","last_became_peered":"2024-09-13T09:17:17.493930+0000","last_unstale":"2024-09-13T09:18:12.043048+0000","last_undegraded":"2024-09-13T09:18:12.043048+0000","last_fullsized":"2024-09-13T09:18:12.043048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:12.092799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000590198,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"61'104","reported_seq":894,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.281582+0000","last_change":"2024-09-13T09:17:40.870470+0000","last_active":"2024-09-13T09:18:13.281582+0000","last_peered":"2024-09-13T09:18:13.281582+0000","last_clean":"2024-09-13T09:18:13.281582+0000","last_became_active":"2024-09-13T09:17:40.793065+0000","last_became_peered":"2024-09-13T09:17:40.793065+0000","last_unstale":"2024-09-13T09:18:13.281582+0000","last_undegraded":"2024-09-13T09:18:13.281582+0000","last_fullsized":"2024-09-13T09:18:13.281582+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:38.586460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052255699999999999,"stat_sum":{"num_bytes":13235581,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":809,"num_read_kb":10955,"num_write":105,"num_write_kb":4986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.062759+0000","last_change":"2024-09-13T09:18:03.968508+0000","last_active":"2024-09-13T09:18:13.062759+0000","last_peered":"2024-09-13T09:18:13.062759+0000","last_clean":"2024-09-13T09:18:13.062759+0000","last_became_active":"2024-09-13T09:18:03.968092+0000","last_became_peered":"2024-09-13T09:18:03.968092+0000","last_unstale":"2024-09-13T09:18:13.062759+0000","last_undegraded":"2024-09-13T09:18:13.062759+0000","last_fullsized":"2024-09-13T09:18:13.062759+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:39.649827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"62'123","reported_seq":999,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.179637+0000","last_change":"2024-09-13T09:17:28.686727+0000","last_active":"2024-09-13T09:18:13.179637+0000","last_peered":"2024-09-13T09:18:13.179637+0000","last_clean":"2024-09-13T09:18:13.179637+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:18:13.179637+0000","last_undegraded":"2024-09-13T09:18:13.179637+0000","last_fullsized":"2024-09-13T09:18:13.179637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:44.094850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061431400000000003,"stat_sum":{"num_bytes":35529168,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":883,"num_read_kb":4408,"num_write":146,"num_write_kb":6730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":58,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.053255+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:18:13.053255+0000","last_peered":"2024-09-13T09:18:13.053255+0000","last_clean":"2024-09-13T09:18:13.053255+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:18:13.053255+0000","last_undegraded":"2024-09-13T09:18:13.053255+0000","last_fullsized":"2024-09-13T09:18:13.053255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.062818+0000","last_change":"2024-09-13T09:17:43.803525+0000","last_active":"2024-09-13T09:18:13.062818+0000","last_peered":"2024-09-13T09:18:13.062818+0000","last_clean":"2024-09-13T09:18:13.062818+0000","last_became_active":"2024-09-13T09:17:43.802603+0000","last_became_peered":"2024-09-13T09:17:43.802603+0000","last_unstale":"2024-09-13T09:18:13.062818+0000","last_undegraded":"2024-09-13T09:18:13.062818+0000","last_fullsized":"2024-09-13T09:18:13.062818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:07.561959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"63'284","reported_seq":758,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:18:14.052766+0000","last_change":"2024-09-13T09:18:04.153662+0000","last_active":"2024-09-13T09:18:14.052766+0000","last_peered":"2024-09-13T09:18:14.052766+0000","last_clean":"2024-09-13T09:18:14.052766+0000","last_became_active":"2024-09-13T09:18:03.974569+0000","last_became_peered":"2024-09-13T09:18:03.974569+0000","last_unstale":"2024-09-13T09:18:14.052766+0000","last_undegraded":"2024-09-13T09:18:14.052766+0000","last_fullsized":"2024-09-13T09:18:14.052766+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:43.479751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014243249999999999,"stat_sum":{"num_bytes":29802270,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":3,"num_read":1644,"num_read_kb":4072,"num_write":408,"num_write_kb":2210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":741496,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"62'296","reported_seq":718,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.761753+0000","last_change":"2024-09-13T09:17:28.681630+0000","last_active":"2024-09-13T09:18:13.761753+0000","last_peered":"2024-09-13T09:18:13.761753+0000","last_clean":"2024-09-13T09:18:13.761753+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:18:13.761753+0000","last_undegraded":"2024-09-13T09:18:13.761753+0000","last_fullsized":"2024-09-13T09:18:13.761753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:06.071033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013508260000000001,"stat_sum":{"num_bytes":5067962,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":1734,"num_read_kb":1918,"num_write":379,"num_write_kb":860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"62'190","reported_seq":1264,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.679698+0000","last_change":"2024-09-13T09:18:06.978086+0000","last_active":"2024-09-13T09:18:13.679698+0000","last_peered":"2024-09-13T09:18:13.679698+0000","last_clean":"2024-09-13T09:18:13.679698+0000","last_became_active":"2024-09-13T09:18:06.976785+0000","last_became_peered":"2024-09-13T09:18:06.976785+0000","last_unstale":"2024-09-13T09:18:13.679698+0000","last_undegraded":"2024-09-13T09:18:13.679698+0000","last_fullsized":"2024-09-13T09:18:13.679698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:19.887646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033158900000000003,"stat_sum":{"num_bytes":11788846,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":1828,"num_read_kb":3991,"num_write":181,"num_write_kb":1188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":387969,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"62'214","reported_seq":499,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.674997+0000","last_change":"2024-09-13T09:17:40.896216+0000","last_active":"2024-09-13T09:18:13.674997+0000","last_peered":"2024-09-13T09:18:13.674997+0000","last_clean":"2024-09-13T09:18:13.674997+0000","last_became_active":"2024-09-13T09:17:40.801195+0000","last_became_peered":"2024-09-13T09:17:40.801195+0000","last_unstale":"2024-09-13T09:18:13.674997+0000","last_undegraded":"2024-09-13T09:18:13.674997+0000","last_fullsized":"2024-09-13T09:18:13.674997+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:00:07.773428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002522725,"stat_sum":{"num_bytes":19534893,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":1015,"num_read_kb":2195,"num_write":305,"num_write_kb":7509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67680,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"62'311","reported_seq":386,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.293071+0000","last_change":"2024-09-13T09:17:28.693070+0000","last_active":"2024-09-13T09:18:13.293071+0000","last_peered":"2024-09-13T09:18:13.293071+0000","last_clean":"2024-09-13T09:18:13.293071+0000","last_became_active":"2024-09-13T09:17:26.698657+0000","last_became_peered":"2024-09-13T09:17:26.698657+0000","last_unstale":"2024-09-13T09:18:13.293071+0000","last_undegraded":"2024-09-13T09:18:13.293071+0000","last_fullsized":"2024-09-13T09:18:13.293071+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:12.388714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002213343,"stat_sum":{"num_bytes":5146854,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":967,"num_read_kb":2066,"num_write":446,"num_write_kb":2568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"62'133","reported_seq":367,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.292367+0000","last_change":"2024-09-13T09:17:28.684477+0000","last_active":"2024-09-13T09:18:13.292367+0000","last_peered":"2024-09-13T09:18:13.292367+0000","last_clean":"2024-09-13T09:18:13.292367+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:18:13.292367+0000","last_undegraded":"2024-09-13T09:18:13.292367+0000","last_fullsized":"2024-09-13T09:18:13.292367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:25.411265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099816899999999992,"stat_sum":{"num_bytes":28558438,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":231,"num_read_kb":1756,"num_write":156,"num_write_kb":7903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"62'186","reported_seq":358,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.179264+0000","last_change":"2024-09-13T09:17:28.687062+0000","last_active":"2024-09-13T09:18:13.179264+0000","last_peered":"2024-09-13T09:18:13.179264+0000","last_clean":"2024-09-13T09:18:13.179264+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:18:13.179264+0000","last_undegraded":"2024-09-13T09:18:13.179264+0000","last_fullsized":"2024-09-13T09:18:13.179264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:31.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034733699999999999,"stat_sum":{"num_bytes":15398067,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":639,"num_read_kb":1993,"num_write":252,"num_write_kb":1359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"62'735","reported_seq":1575,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.276278+0000","last_change":"2024-09-13T09:17:28.684481+0000","last_active":"2024-09-13T09:18:13.276278+0000","last_peered":"2024-09-13T09:18:13.276278+0000","last_clean":"2024-09-13T09:18:13.276278+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:18:13.276278+0000","last_undegraded":"2024-09-13T09:18:13.276278+0000","last_fullsized":"2024-09-13T09:18:13.276278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:56.423075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151104,"stat_sum":{"num_bytes":12578626,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":4199,"num_read_kb":5638,"num_write":1229,"num_write_kb":1658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"62'108","reported_seq":296,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.521703+0000","last_change":"2024-09-13T09:17:57.986034+0000","last_active":"2024-09-13T09:18:13.521703+0000","last_peered":"2024-09-13T09:18:13.521703+0000","last_clean":"2024-09-13T09:18:13.521703+0000","last_became_active":"2024-09-13T09:17:57.919132+0000","last_became_peered":"2024-09-13T09:17:57.919132+0000","last_unstale":"2024-09-13T09:18:13.521703+0000","last_undegraded":"2024-09-13T09:18:13.521703+0000","last_fullsized":"2024-09-13T09:18:13.521703+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:49.597931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037557999999999998,"stat_sum":{"num_bytes":3588316,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":216,"num_read_kb":455,"num_write":123,"num_write_kb":778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":20268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"62'91","reported_seq":594,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.724975+0000","last_change":"2024-09-13T09:17:28.683788+0000","last_active":"2024-09-13T09:18:13.724975+0000","last_peered":"2024-09-13T09:18:13.724975+0000","last_clean":"2024-09-13T09:18:13.724975+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:18:13.724975+0000","last_undegraded":"2024-09-13T09:18:13.724975+0000","last_fullsized":"2024-09-13T09:18:13.724975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:47.572839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070075400000000005,"stat_sum":{"num_bytes":6598909,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":476,"num_read_kb":3271,"num_write":89,"num_write_kb":1343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"63'193","reported_seq":1218,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:18:14.082795+0000","last_change":"2024-09-13T09:17:40.816999+0000","last_active":"2024-09-13T09:18:14.082795+0000","last_peered":"2024-09-13T09:18:14.082795+0000","last_clean":"2024-09-13T09:18:14.082795+0000","last_became_active":"2024-09-13T09:17:40.789499+0000","last_became_peered":"2024-09-13T09:17:40.789499+0000","last_unstale":"2024-09-13T09:18:14.082795+0000","last_undegraded":"2024-09-13T09:18:14.082795+0000","last_fullsized":"2024-09-13T09:18:14.082795+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:13.395268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084252600000000002,"stat_sum":{"num_bytes":23315181,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2062,"num_read_kb":4921,"num_write":200,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1322,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"62'84","reported_seq":891,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.768042+0000","last_change":"2024-09-13T09:17:28.683729+0000","last_active":"2024-09-13T09:18:13.768042+0000","last_peered":"2024-09-13T09:18:13.768042+0000","last_clean":"2024-09-13T09:18:13.768042+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:18:13.768042+0000","last_undegraded":"2024-09-13T09:18:13.768042+0000","last_fullsized":"2024-09-13T09:18:13.768042+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:07.893848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000594467,"stat_sum":{"num_bytes":12127016,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":794,"num_read_kb":3562,"num_write":79,"num_write_kb":926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":520170916,"num_objects":561,"num_object_clones":99,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":135,"num_read":41233,"num_read_kb":108640,"num_write":7824,"num_write_kb":85996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":577,"num_bytes_recovered":4820954,"num_keys_recovered":127,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19865600,"data_stored":115547127,"data_compressed":1456179,"data_compressed_allocated":7749632,"data_compressed_original":105436591,"omap_allocated":0,"internal_metadata":0},"log_size":6072,"ondisk_log_size":6072,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":30095,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42764,"kb_used_data":3452,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94329076,"statfs":{"total":96636764160,"available":96592973824,"internally_reserved":0,"allocated":3534848,"data_stored":17879189,"data_compressed":168901,"data_compressed_allocated":1150976,"data_compressed_original":16203776,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.2360000000000002,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.873}]},{"osd":1,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.92700000000000005},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":3.9039999999999999,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.81200000000000006}]},{"osd":3,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.94799999999999995}]},{"osd":4,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":3.6139999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.625}]},{"osd":5,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.125,"5min":3.125,"15min":3.125},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.3130000000000002,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.68100000000000005}]}]},{"osd":6,"up_from":0,"seq":55834574864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44180,"kb_used_data":4844,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327660,"statfs":{"total":96636764160,"available":96591523840,"internally_reserved":0,"allocated":4960256,"data_stored":31922981,"data_compressed":315680,"data_compressed_allocated":2048000,"data_compressed_original":29736960,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":7.8150000000000004,"5min":7.8150000000000004,"15min":7.8150000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":132.32900000000001,"5min":132.32900000000001,"15min":132.32900000000001},"last":1.091},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.236}]},{"osd":1,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.4419999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":8.2629999999999999,"5min":8.2629999999999999,"15min":8.2629999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":132.23599999999999,"5min":132.23599999999999,"15min":132.23599999999999},"last":1.284}]},{"osd":6,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.0569999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.383},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.5910000000000002,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.6379999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31000,"kb_used_data":3976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340840,"statfs":{"total":96636764160,"available":96605020160,"internally_reserved":0,"allocated":4071424,"data_stored":21224840,"data_compressed":299835,"data_compressed_allocated":1462272,"data_compressed_original":19320832,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.77600000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.95199999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.331}]},{"osd":5,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.92300000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.165},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.2789999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.2470000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43216,"kb_used_data":3912,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94328624,"statfs":{"total":96636764160,"available":96592510976,"internally_reserved":0,"allocated":4005888,"data_stored":19581105,"data_compressed":240462,"data_compressed_allocated":1269760,"data_compressed_original":17603621,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":1.5720000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.282},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.89500000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.544},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.68100000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.669},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.1619999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.22}]},{"osd":6,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.347},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":0.96699999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":15.4,"5min":15.4,"15min":15.4},"last":1.47}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36724,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335116,"statfs":{"total":96636764160,"available":96599158784,"internally_reserved":0,"allocated":1527808,"data_stored":1718650,"data_compressed":33249,"data_compressed_allocated":86016,"data_compressed_original":802816,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.67600000000000005}]},{"osd":3,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.78800000000000003}]},{"osd":5,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.9809999999999999,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.85399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37536,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334304,"statfs":{"total":96636764160,"available":96598327296,"internally_reserved":0,"allocated":2379776,"data_stored":4044667,"data_compressed":98758,"data_compressed_allocated":286720,"data_compressed_original":2632074,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.47199999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.42699999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":8.3460000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.71099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39260,"kb_used_data":4020,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332580,"statfs":{"total":96636764160,"available":96596561920,"internally_reserved":0,"allocated":4116480,"data_stored":21839273,"data_compressed":287339,"data_compressed_allocated":1454080,"data_compressed_original":19824640,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.40699999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.58099999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.623},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.246}]},{"osd":5,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.151},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.111}]},{"osd":6,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.018},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.82499999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":0.95699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3358720,"data_stored":20929682,"data_compressed":276961,"data_compressed_allocated":1417216,"data_compressed_original":19234816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1642496,"data_stored":3718522,"data_compressed":98758,"data_compressed_allocated":286720,"data_compressed_original":2632074,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":901120,"data_stored":1501575,"data_compressed":33396,"data_compressed_allocated":90112,"data_compressed_original":831488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3284992,"data_stored":19261127,"data_compressed":240462,"data_compressed_allocated":1269760,"data_compressed_original":17603621,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3354624,"data_stored":20904946,"data_compressed":299835,"data_compressed_allocated":1462272,"data_compressed_original":19320832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4280320,"data_stored":31629138,"data_compressed":315680,"data_compressed_allocated":2048000,"data_compressed_original":29736960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":609108,"data_compressed":31525,"data_compressed_allocated":61440,"data_compressed_original":446464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2785280,"data_stored":16993029,"data_compressed":159562,"data_compressed_allocated":1114112,"data_compressed_original":15630336,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:18:15.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:18:15.515 INFO:teuthology.orchestra.run.smithi032.stdout:2769 writesame 0x8262599 thru 0x8269532 data_size 0xa(0x6f9a bytes) 2024-09-13T09:18:15.518 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:15.514+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:15.523 INFO:teuthology.orchestra.run.smithi032.stdout:2770 punch from 0x49bdc2a to 0x49c5444, (0x781a bytes) 2024-09-13T09:18:15.524 INFO:teuthology.orchestra.run.smithi032.stdout:2771 read 0xc4ab584 thru 0xc4aee0a (0x3887 bytes) 2024-09-13T09:18:15.524 INFO:teuthology.orchestra.run.smithi032.stdout:2772 write 0xcd563cf thru 0xcd5eee6 (0x8b18 bytes) 2024-09-13T09:18:15.529 INFO:teuthology.orchestra.run.smithi032.stdout:2773 punch from 0x5c651ce to 0x5c68140, (0x2f72 bytes) 2024-09-13T09:18:15.530 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:15.530 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:15.530 INFO:teuthology.orchestra.run.smithi032.stdout:2777 write 0xb5a67ac thru 0xb5b5332 (0xeb87 bytes) 2024-09-13T09:18:15.537 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:15.537 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:15.537 INFO:teuthology.orchestra.run.smithi032.stdout:2780 write 0x2960266 thru 0x2969b54 (0x98ef bytes) 2024-09-13T09:18:15.547 INFO:teuthology.orchestra.run.smithi032.stdout:2781 read 0xd42b947 thru 0xd439d2c (0xe3e6 bytes) 2024-09-13T09:18:15.547 INFO:teuthology.orchestra.run.smithi032.stdout:2782 write 0x5bbce4f thru 0x5bcb002 (0xe1b4 bytes) 2024-09-13T09:18:15.558 INFO:teuthology.orchestra.run.smithi032.stdout:2783 write 0xa0cf0ef thru 0xa0d4093 (0x4fa5 bytes) 2024-09-13T09:18:15.566 INFO:teuthology.orchestra.run.smithi032.stdout:2785 write 0x4b53b77 thru 0x4b5cad4 (0x8f5e bytes) 2024-09-13T09:18:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:2786 write 0xa047590 thru 0xa047e2c (0x89d bytes) 2024-09-13T09:18:15.579 INFO:teuthology.orchestra.run.smithi032.stdout:2787 read 0x8b5a0f2 thru 0x8b66e72 (0xcd81 bytes) 2024-09-13T09:18:15.580 INFO:teuthology.orchestra.run.smithi032.stdout:2789 punch from 0x70891ca to 0x708f250, (0x6086 bytes) 2024-09-13T09:18:15.580 INFO:teuthology.orchestra.run.smithi032.stdout:2790 writesame 0x807f1fd thru 0x808a801 data_size 0x1(0xb605 bytes) 2024-09-13T09:18:15.618 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:15.619+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:15.635 INFO:teuthology.orchestra.run.smithi032.stdout:2791 read 0x1d8ddd1 thru 0x1d91b7d (0x3dad bytes) 2024-09-13T09:18:15.635 INFO:teuthology.orchestra.run.smithi032.stdout:2792 read 0xcbb918b thru 0xcbc22af (0x9125 bytes) 2024-09-13T09:18:15.635 INFO:teuthology.orchestra.run.smithi032.stdout:2793 write 0x78f6a54 thru 0x79003b7 (0x9964 bytes) 2024-09-13T09:18:15.639 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:15.639 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:15.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:15.639 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:15.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.640 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:3323 3.4 e35/22 3:2630d0ac:::rbd_data.118591b29fad.000000000000007a:head v 35'34, mlcod=35'31)", 2024-09-13T09:18:15.640 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:18:15.640 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.103280339000001, 2024-09-13T09:18:15.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037621640000000001, 2024-09-13T09:18:15.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.640 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.640 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:18:15.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535795+0000", 2024-09-13T09:18:15.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535793+0000", 2024-09-13T09:18:15.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:15.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535811+0000", 2024-09-13T09:18:15.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8170000000000001e-05 2024-09-13T09:18:15.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535840+0000", 2024-09-13T09:18:15.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9105999999999999e-05 2024-09-13T09:18:15.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.535877+0000", 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7017000000000002e-05 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536001+0000", 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012342899999999999 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.536073+0000", 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2123e-05 2024-09-13T09:18:15.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539433+0000", 2024-09-13T09:18:15.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033596630000000001 2024-09-13T09:18:15.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.539557+0000", 2024-09-13T09:18:15.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012470600000000001 2024-09-13T09:18:15.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.646 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.647 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16010 3.4 3:27c24f46:::rbd_data.11856af9a4d3.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:15.647 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:18:15.647 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.638402033999999, 2024-09-13T09:18:15.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037539579999999999, 2024-09-13T09:18:15.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.647 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.647 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16010 2024-09-13T09:18:15.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.648 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:18:15.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000673+0000", 2024-09-13T09:18:15.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000672+0000", 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000679+0000", 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9120000000000001e-06 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000693+0000", 2024-09-13T09:18:15.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3745e-05 2024-09-13T09:18:15.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000702+0000", 2024-09-13T09:18:15.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.064e-06 2024-09-13T09:18:15.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000763+0000", 2024-09-13T09:18:15.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1309000000000005e-05 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.000877+0000", 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114214 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.001145+0000", 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000267765 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.002514+0000", 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013688730000000001 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004301+0000", 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017869839999999999 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004380+0000", 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8861000000000003e-05 2024-09-13T09:18:15.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.004427+0000", 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7713000000000002e-05 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16293 3.4 3:22419c10:::rbd_data.1185f20bdb42.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.256914247000001, 2024-09-13T09:18:15.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015655326000000001, 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16293 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382161+0000", 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382158+0000", 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382176+0000", 2024-09-13T09:18:15.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7863000000000002e-05 2024-09-13T09:18:15.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382211+0000", 2024-09-13T09:18:15.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.481e-05 2024-09-13T09:18:15.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382226+0000", 2024-09-13T09:18:15.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.537e-05 2024-09-13T09:18:15.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382354+0000", 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012803599999999999 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.382591+0000", 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023629099999999999 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.385269+0000", 2024-09-13T09:18:15.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002678532 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397515+0000", 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012245946000000001 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397551+0000", 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5856999999999999e-05 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397699+0000", 2024-09-13T09:18:15.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148255 2024-09-13T09:18:15.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397817+0000", 2024-09-13T09:18:15.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117299 2024-09-13T09:18:15.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.659 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.660 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.660 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4485.0:16293 3.4 e42/41 ondisk, result = 0)", 2024-09-13T09:18:15.660 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:18:15.660 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.246582707000002, 2024-09-13T09:18:15.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0050343319999999999, 2024-09-13T09:18:15.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.660 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.660 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:18:15.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392493+0000", 2024-09-13T09:18:15.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392491+0000", 2024-09-13T09:18:15.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:15.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392500+0000", 2024-09-13T09:18:15.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8300000000000002e-06 2024-09-13T09:18:15.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392529+0000", 2024-09-13T09:18:15.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9260000000000001e-05 2024-09-13T09:18:15.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.392542+0000", 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2969999999999999e-05 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397468+0000", 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0049256880000000001 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397496+0000", 2024-09-13T09:18:15.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8209999999999999e-05 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:52.397527+0000", 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1276999999999999e-05 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:16837 3.0 3:01a39f05:::rbd_data.11859d3d11dd.000000000000000a:head [call rbd.sparse_copyup in=173207b,set-alloc-hint object_size 16777216 write_size 16777216,write 3145728~10909 in=10909b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:18:15.664 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.378951894, 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004073887, 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 16837 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:18:15.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260124+0000", 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260121+0000", 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260266+0000", 2024-09-13T09:18:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014495999999999999 2024-09-13T09:18:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260287+0000", 2024-09-13T09:18:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1336e-05 2024-09-13T09:18:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260296+0000", 2024-09-13T09:18:15.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6170000000000007e-06 2024-09-13T09:18:15.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260352+0000", 2024-09-13T09:18:15.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6252999999999998e-05 2024-09-13T09:18:15.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260520+0000", 2024-09-13T09:18:15.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000168381 2024-09-13T09:18:15.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:18:15.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.260807+0000", 2024-09-13T09:18:15.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028624100000000001 2024-09-13T09:18:15.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.262062+0000", 2024-09-13T09:18:15.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012551490000000001 2024-09-13T09:18:15.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264128+0000", 2024-09-13T09:18:15.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020660380000000001 2024-09-13T09:18:15.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264167+0000", 2024-09-13T09:18:15.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9714e-05 2024-09-13T09:18:15.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:53.264198+0000", 2024-09-13T09:18:15.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0105000000000001e-05 2024-09-13T09:18:15.670 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:18757 3.4 3:21c7a0ac:::rbd_data.1185b72576e4.0000000000000074:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.009041092, 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066789190000000002, 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 18757 2024-09-13T09:18:15.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.672 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:18:15.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630034+0000", 2024-09-13T09:18:15.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630032+0000", 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630044+0000", 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2245999999999999e-05 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630067+0000", 2024-09-13T09:18:15.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3404000000000002e-05 2024-09-13T09:18:15.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630078+0000", 2024-09-13T09:18:15.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1158999999999999e-05 2024-09-13T09:18:15.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630156+0000", 2024-09-13T09:18:15.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7520000000000003e-05 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.630357+0000", 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020064700000000001 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.631878+0000", 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015212940000000001 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636574+0000", 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046960719999999999 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636612+0000", 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7499999999999997e-05 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636669+0000", 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7286e-05 2024-09-13T09:18:15.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.636713+0000", 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4530000000000002e-05 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:20630 3.4 3:23dc35fd:::rbd_data.1185b72576e4.000000000000007c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:00.986895+0000", 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.652180669, 2024-09-13T09:18:15.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0040477539999999998, 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 20630 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986895+0000", 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986895+0000", 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986893+0000", 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986899+0000", 2024-09-13T09:18:15.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-13T09:18:15.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986910+0000", 2024-09-13T09:18:15.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1105e-05 2024-09-13T09:18:15.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986917+0000", 2024-09-13T09:18:15.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:18:15.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.986963+0000", 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5932e-05 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.987020+0000", 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6858999999999999e-05 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.987104+0000", 2024-09-13T09:18:15.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4250000000000001e-05 2024-09-13T09:18:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.987786+0000", 2024-09-13T09:18:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00068249499999999998 2024-09-13T09:18:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.990828+0000", 2024-09-13T09:18:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030412099999999999 2024-09-13T09:18:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.990893+0000", 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5117999999999996e-05 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:00.990943+0000", 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9789000000000001e-05 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:21345 3.0 3:004c516c:::rbd_header.1185d6f8ec87:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:02.094060+0000", 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.545015693, 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004611782, 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 21345 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094060+0000", 2024-09-13T09:18:15.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094060+0000", 2024-09-13T09:18:15.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094057+0000", 2024-09-13T09:18:15.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:15.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094070+0000", 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2787e-05 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094095+0000", 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4873000000000001e-05 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094106+0000", 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0954e-05 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094171+0000", 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4958999999999999e-05 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094344+0000", 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000173098 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.094590+0000", 2024-09-13T09:18:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024549199999999999 2024-09-13T09:18:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.095579+0000", 2024-09-13T09:18:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00098965800000000003 2024-09-13T09:18:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.098585+0000", 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030058390000000002 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.098630+0000", 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5393999999999999e-05 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:02.098672+0000", 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1153999999999999e-05 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.689 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:24733 3.6 3:673d20ef:::rbd_data.118513c288ae.000000000000002e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:10.357682+0000", 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.281393091, 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013452253000000001, 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 24733 2024-09-13T09:18:15.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357682+0000", 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357682+0000", 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357681+0000", 2024-09-13T09:18:15.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:15.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357690+0000", 2024-09-13T09:18:15.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:18:15.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357705+0000", 2024-09-13T09:18:15.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4847e-05 2024-09-13T09:18:15.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357713+0000", 2024-09-13T09:18:15.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.469e-06 2024-09-13T09:18:15.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357776+0000", 2024-09-13T09:18:15.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3490000000000004e-05 2024-09-13T09:18:15.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.357937+0000", 2024-09-13T09:18:15.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161077 2024-09-13T09:18:15.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:18:15.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.360165+0000", 2024-09-13T09:18:15.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022276330000000001 2024-09-13T09:18:15.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.371006+0000", 2024-09-13T09:18:15.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010841046999999999 2024-09-13T09:18:15.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.371023+0000", 2024-09-13T09:18:15.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6872e-05 2024-09-13T09:18:15.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.371083+0000", 2024-09-13T09:18:15.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9991999999999998e-05 2024-09-13T09:18:15.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.371135+0000", 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2064000000000002e-05 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4485.0:24733 3.6 e59/52 ondisk, result = 0)", 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:10.366395+0000", 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.2726809299999999, 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0046156160000000003, 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.696 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.366395+0000", 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.366395+0000", 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.366394+0000", 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.366398+0000", 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.335e-06 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.366412+0000", 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3879e-05 2024-09-13T09:18:15.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.366419+0000", 2024-09-13T09:18:15.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9319999999999999e-06 2024-09-13T09:18:15.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.370987+0000", 2024-09-13T09:18:15.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045677410000000002 2024-09-13T09:18:15.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.370999+0000", 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1884e-05 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.371010+0000", 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:24735 3.6 3:673d20ef:::rbd_data.118513c288ae.000000000000002e:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-13T09:18:15.700 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:10.372012+0000", 2024-09-13T09:18:15.701 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.2670633650000003, 2024-09-13T09:18:15.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0053273979999999997, 2024-09-13T09:18:15.701 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.701 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.701 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.701 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.701 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.701 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 24735 2024-09-13T09:18:15.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.701 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372012+0000", 2024-09-13T09:18:15.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372012+0000", 2024-09-13T09:18:15.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372011+0000", 2024-09-13T09:18:15.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:15.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372017+0000", 2024-09-13T09:18:15.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.331e-06 2024-09-13T09:18:15.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372030+0000", 2024-09-13T09:18:15.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2203e-05 2024-09-13T09:18:15.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372036+0000", 2024-09-13T09:18:15.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2999999999999998e-06 2024-09-13T09:18:15.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372071+0000", 2024-09-13T09:18:15.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.54e-05 2024-09-13T09:18:15.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372124+0000", 2024-09-13T09:18:15.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2621000000000003e-05 2024-09-13T09:18:15.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:18:15.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.372908+0000", 2024-09-13T09:18:15.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00078428899999999997 2024-09-13T09:18:15.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.377224+0000", 2024-09-13T09:18:15.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0043153510000000003 2024-09-13T09:18:15.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.377238+0000", 2024-09-13T09:18:15.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4114000000000001e-05 2024-09-13T09:18:15.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.377291+0000", 2024-09-13T09:18:15.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3823999999999998e-05 2024-09-13T09:18:15.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:10.377340+0000", 2024-09-13T09:18:15.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8115999999999998e-05 2024-09-13T09:18:15.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.707 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.707 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:26866 3.6 3:666c541f:::rbd_object_map.118555ddeb59.0000000000000015:head [writefull 0~92] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:18:15.707 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:15.068371+0000", 2024-09-13T09:18:15.707 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.57070438199999995, 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037767769999999998, 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 26866 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.068371+0000", 2024-09-13T09:18:15.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.068371+0000", 2024-09-13T09:18:15.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.068369+0000", 2024-09-13T09:18:15.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:15.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.068385+0000", 2024-09-13T09:18:15.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5965e-05 2024-09-13T09:18:15.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.068402+0000", 2024-09-13T09:18:15.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6974e-05 2024-09-13T09:18:15.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.068410+0000", 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0199999999999994e-06 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.068459+0000", 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8622999999999998e-05 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.068464+0000", 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.070340+0000", 2024-09-13T09:18:15.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001875514 2024-09-13T09:18:15.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.070508+0000", 2024-09-13T09:18:15.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000168708 2024-09-13T09:18:15.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:18:15.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.070662+0000", 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000153332 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.071473+0000", 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00081183299999999998 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.072045+0000", 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00057112099999999996 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.072103+0000", 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8050000000000002e-05 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.072148+0000", 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5299000000000001e-05 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.715 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:27023 3.6 3:666c541f:::rbd_object_map.118555ddeb59.0000000000000015:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:18:15.715 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:15.204765+0000", 2024-09-13T09:18:15.715 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.434310529, 2024-09-13T09:18:15.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0048279500000000001, 2024-09-13T09:18:15.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.715 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.715 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 27023 2024-09-13T09:18:15.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.715 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.204765+0000", 2024-09-13T09:18:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.204765+0000", 2024-09-13T09:18:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.204763+0000", 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.204773+0000", 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6719999999999999e-06 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.204793+0000", 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0285e-05 2024-09-13T09:18:15.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.204801+0000", 2024-09-13T09:18:15.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6290000000000001e-06 2024-09-13T09:18:15.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.204843+0000", 2024-09-13T09:18:15.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1974999999999997e-05 2024-09-13T09:18:15.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.204941+0000", 2024-09-13T09:18:15.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8485000000000003e-05 2024-09-13T09:18:15.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:18:15.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.205317+0000", 2024-09-13T09:18:15.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037552700000000001 2024-09-13T09:18:15.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208833+0000", 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035163709999999999 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.209529+0000", 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00069596700000000003 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.209559+0000", 2024-09-13T09:18:15.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9541999999999999e-05 2024-09-13T09:18:15.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.209593+0000", 2024-09-13T09:18:15.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4167e-05 2024-09-13T09:18:15.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.721 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.721 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:27027 3.6 3:666c541f:::rbd_object_map.118555ddeb59.0000000000000015:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:18:15.721 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:15.207641+0000", 2024-09-13T09:18:15.722 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.431434763, 2024-09-13T09:18:15.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062790779999999996, 2024-09-13T09:18:15.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.722 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.722 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 27027 2024-09-13T09:18:15.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.722 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.207641+0000", 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.207641+0000", 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.207638+0000", 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.207649+0000", 2024-09-13T09:18:15.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0281e-05 2024-09-13T09:18:15.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.207663+0000", 2024-09-13T09:18:15.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4842e-05 2024-09-13T09:18:15.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.207672+0000", 2024-09-13T09:18:15.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1240000000000005e-06 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.207811+0000", 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000139524 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208015+0000", 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000204038 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:18:15.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208402+0000", 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038729500000000001 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.213420+0000", 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0050171870000000002 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.213683+0000", 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026307000000000001 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.213814+0000", 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013160299999999999 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.213920+0000", 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105542 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:27028 3.6 3:666c541f:::rbd_object_map.118555ddeb59.0000000000000015:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:18:15.727 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:15.208027+0000", 2024-09-13T09:18:15.728 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.43104852399999999, 2024-09-13T09:18:15.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0061774600000000001, 2024-09-13T09:18:15.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.728 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.728 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.728 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.728 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.728 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 27028 2024-09-13T09:18:15.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.728 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208027+0000", 2024-09-13T09:18:15.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208027+0000", 2024-09-13T09:18:15.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208024+0000", 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208042+0000", 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7221999999999999e-05 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208078+0000", 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6334999999999997e-05 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208099+0000", 2024-09-13T09:18:15.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1101e-05 2024-09-13T09:18:15.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208869+0000", 2024-09-13T09:18:15.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00077004700000000005 2024-09-13T09:18:15.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208942+0000", 2024-09-13T09:18:15.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2420000000000001e-05 2024-09-13T09:18:15.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:18:15.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.209228+0000", 2024-09-13T09:18:15.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000286364 2024-09-13T09:18:15.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.213452+0000", 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042243180000000003 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.214124+0000", 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00067184500000000002 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.214169+0000", 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4712999999999997e-05 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.214204+0000", 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5605000000000002e-05 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:27029 3.6 3:666c541f:::rbd_object_map.118555ddeb59.0000000000000015:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:15.208573+0000", 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.43050225199999997, 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0057043329999999998, 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 27029 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208573+0000", 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208573+0000", 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208571+0000", 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208587+0000", 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6252e-05 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208627+0000", 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9718000000000002e-05 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208651+0000", 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4535000000000001e-05 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.209659+0000", 2024-09-13T09:18:15.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001007628 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.209736+0000", 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6885000000000007e-05 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.210035+0000", 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000299303 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.213483+0000", 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003447978 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.214216+0000", 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00073310400000000005 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.214245+0000", 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8983999999999999e-05 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.214278+0000", 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2273000000000001e-05 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:27031 3.6 3:666c541f:::rbd_object_map.118555ddeb59.0000000000000015:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:15.208691+0000", 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.43038483399999999, 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0056696009999999998, 2024-09-13T09:18:15.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.741 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.741 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 27031 2024-09-13T09:18:15.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.741 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208691+0000", 2024-09-13T09:18:15.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208691+0000", 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208690+0000", 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208696+0000", 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1469999999999998e-06 2024-09-13T09:18:15.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208710+0000", 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4392999999999999e-05 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.208729+0000", 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8136000000000001e-05 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.210509+0000", 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001780018 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.210584+0000", 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5735000000000006e-05 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.210880+0000", 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029587100000000001 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.213648+0000", 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027673179999999999 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.214289+0000", 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00064152999999999996 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.214330+0000", 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0778000000000002e-05 2024-09-13T09:18:15.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.214360+0000", 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0474999999999999e-05 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:27041 3.6 3:666c541f:::rbd_object_map.118555ddeb59.0000000000000015:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:15.210719+0000", 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.42835673200000002, 2024-09-13T09:18:15.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077436470000000002, 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 27041 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.210719+0000", 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.210719+0000", 2024-09-13T09:18:15.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.210716+0000", 2024-09-13T09:18:15.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:15.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.210733+0000", 2024-09-13T09:18:15.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6541999999999998e-05 2024-09-13T09:18:15.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.210766+0000", 2024-09-13T09:18:15.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2978000000000002e-05 2024-09-13T09:18:15.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.210787+0000", 2024-09-13T09:18:15.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1237999999999999e-05 2024-09-13T09:18:15.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.212796+0000", 2024-09-13T09:18:15.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002008503 2024-09-13T09:18:15.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.212881+0000", 2024-09-13T09:18:15.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5261999999999998e-05 2024-09-13T09:18:15.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:18:15.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.213115+0000", 2024-09-13T09:18:15.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023405600000000001 2024-09-13T09:18:15.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:15.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.214582+0000", 2024-09-13T09:18:15.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014666220000000001 2024-09-13T09:18:15.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:15.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.218371+0000", 2024-09-13T09:18:15.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037896570000000001 2024-09-13T09:18:15.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.218422+0000", 2024-09-13T09:18:15.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0651e-05 2024-09-13T09:18:15.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.218462+0000", 2024-09-13T09:18:15.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0611000000000001e-05 2024-09-13T09:18:15.753 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.753 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.753 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.753 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:27038 3.0 e64/52 3:05667292:::rbd_data.118555ddeb59.0000000000000008:head v 64'193, mlcod=64'191)", 2024-09-13T09:18:15.753 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:15.211436+0000", 2024-09-13T09:18:15.753 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.42763904400000002, 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.006989533, 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.211436+0000", 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.211436+0000", 2024-09-13T09:18:15.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.211435+0000", 2024-09-13T09:18:15.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:15.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.211733+0000", 2024-09-13T09:18:15.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029828999999999999 2024-09-13T09:18:15.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.211769+0000", 2024-09-13T09:18:15.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6186000000000002e-05 2024-09-13T09:18:15.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.211881+0000", 2024-09-13T09:18:15.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111524 2024-09-13T09:18:15.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.212069+0000", 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018833500000000001 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.212089+0000", 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9672000000000001e-05 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.218376+0000", 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062872329999999997 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.218426+0000", 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9969999999999998e-05 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4485.0:27039 3.0 e64/52 3:05667292:::rbd_data.118555ddeb59.0000000000000008:head v 64'195, mlcod=64'191)", 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:15.212824+0000", 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.42625194399999999, 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062591729999999998, 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:15.758 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:15.759 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:15.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:15.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.212824+0000", 2024-09-13T09:18:15.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:15.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.212824+0000", 2024-09-13T09:18:15.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:15.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:15.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.212823+0000", 2024-09-13T09:18:15.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:15.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:15.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.212828+0000", 2024-09-13T09:18:15.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4800000000000003e-06 2024-09-13T09:18:15.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:15.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.212834+0000", 2024-09-13T09:18:15.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-13T09:18:15.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:15.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.212838+0000", 2024-09-13T09:18:15.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2910000000000001e-06 2024-09-13T09:18:15.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:15.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.212924+0000", 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6204000000000003e-05 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.212945+0000", 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.037e-05 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.219046+0000", 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0061013209999999998 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:15.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:15.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:15.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:15.219083+0000", 2024-09-13T09:18:15.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6681999999999999e-05 2024-09-13T09:18:15.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:15.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:15.763 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:15.764 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:15.764 INFO:teuthology.orchestra.run.smithi032.stdout:2796 write 0x7a5b6a0 thru 0x7a6acf5 (0xf656 bytes) 2024-09-13T09:18:15.765 INFO:teuthology.orchestra.run.smithi032.stdout:2797 read 0x4c699b9 thru 0x4c764b2 (0xcafa bytes) 2024-09-13T09:18:15.765 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:15.766 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:15.766 INFO:teuthology.orchestra.run.smithi032.stdout:2799 trunc from 0xd4aee5d to 0xe848fd9 2024-09-13T09:18:15.767 INFO:teuthology.orchestra.run.smithi032.stdout:2800 trunc from 0xe848fd9 to 0xbc6ca28 2024-09-13T09:18:15.767 INFO:teuthology.orchestra.run.smithi032.stdout:2801 write 0x6d1f9ae thru 0x6d24a88 (0x50db bytes) 2024-09-13T09:18:15.767 INFO:teuthology.orchestra.run.smithi032.stdout:2802 read 0x42ddbbb thru 0x42ec85e (0xeca4 bytes) 2024-09-13T09:18:15.767 INFO:teuthology.orchestra.run.smithi032.stdout:2803 write 0xc7fb8c2 thru 0xc804fd2 (0x9711 bytes) 2024-09-13T09:18:15.767 INFO:teuthology.orchestra.run.smithi032.stdout:2804 clone 19 order 20 su 4096 sc 7 2024-09-13T09:18:15.767 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:15.758+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:15.827 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:18:15.828 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:18:15.845 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-09-13T09:18:15.055552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":520761303,"num_objects":565,"num_object_clones":99,"num_object_copies":1130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":565,"num_whiteouts":135,"num_read":41349,"num_read_kb":108740,"num_write":7985,"num_write_kb":88414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":6001747,"num_keys_recovered":127,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6192,"ondisk_log_size":6192,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":274680,"kb_used_data":24020,"kb_used_omap":10,"kb_used_meta":250485,"kb_avail":660328200,"statfs":{"total":676457349120,"available":676176076800,"internally_reserved":0,"allocated":24596480,"data_stored":118210705,"data_compressed":1444224,"data_compressed_allocated":7757824,"data_compressed_original":106124719,"omap_allocated":11151,"internal_metadata":256496753},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":107783787,"num_objects":97,"num_object_clones":6,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":17,"num_read":4277,"num_read_kb":7130,"num_write":966,"num_write_kb":8071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040681"},"pg_stats":[{"pgid":"3.1f","version":"62'126","reported_seq":252,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.608782+0000","last_change":"2024-09-13T09:17:40.849739+0000","last_active":"2024-09-13T09:18:13.608782+0000","last_peered":"2024-09-13T09:18:13.608782+0000","last_clean":"2024-09-13T09:18:13.608782+0000","last_became_active":"2024-09-13T09:17:40.791187+0000","last_became_peered":"2024-09-13T09:17:40.791187+0000","last_unstale":"2024-09-13T09:18:13.608782+0000","last_undegraded":"2024-09-13T09:18:13.608782+0000","last_fullsized":"2024-09-13T09:18:13.608782+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:13.280600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046778999999999999,"stat_sum":{"num_bytes":25108591,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":383,"num_read_kb":3309,"num_write":201,"num_write_kb":8251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"61'354","reported_seq":526,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.045866+0000","last_change":"2024-09-13T09:18:00.931799+0000","last_active":"2024-09-13T09:18:13.045866+0000","last_peered":"2024-09-13T09:18:13.045866+0000","last_clean":"2024-09-13T09:18:13.045866+0000","last_became_active":"2024-09-13T09:18:00.931068+0000","last_became_peered":"2024-09-13T09:18:00.931068+0000","last_unstale":"2024-09-13T09:18:13.045866+0000","last_undegraded":"2024-09-13T09:18:13.045866+0000","last_fullsized":"2024-09-13T09:18:13.045866+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:39:18.748434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382264,"stat_sum":{"num_bytes":10875609,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":1406,"num_read_kb":1836,"num_write":552,"num_write_kb":1683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":412132,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"63'162","reported_seq":854,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:18:14.082721+0000","last_change":"2024-09-13T09:17:28.692127+0000","last_active":"2024-09-13T09:18:14.082721+0000","last_peered":"2024-09-13T09:18:14.082721+0000","last_clean":"2024-09-13T09:18:14.082721+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:18:14.082721+0000","last_undegraded":"2024-09-13T09:18:14.082721+0000","last_fullsized":"2024-09-13T09:18:14.082721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:20.509991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001448682,"stat_sum":{"num_bytes":20453247,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":1889,"num_read_kb":2719,"num_write":192,"num_write_kb":8135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"63'332","reported_seq":1511,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:18:14.129970+0000","last_change":"2024-09-13T09:17:28.691986+0000","last_active":"2024-09-13T09:18:14.129970+0000","last_peered":"2024-09-13T09:18:14.129970+0000","last_clean":"2024-09-13T09:18:14.129970+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:18:14.129970+0000","last_undegraded":"2024-09-13T09:18:14.129970+0000","last_fullsized":"2024-09-13T09:18:14.129970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:05.002231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013280939999999999,"stat_sum":{"num_bytes":1839020,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":3758,"num_read_kb":3417,"num_write":407,"num_write_kb":1380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"62'337","reported_seq":1943,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.288877+0000","last_change":"2024-09-13T09:17:40.897770+0000","last_active":"2024-09-13T09:18:13.288877+0000","last_peered":"2024-09-13T09:18:13.288877+0000","last_clean":"2024-09-13T09:18:13.288877+0000","last_became_active":"2024-09-13T09:17:40.799618+0000","last_became_peered":"2024-09-13T09:17:40.799618+0000","last_unstale":"2024-09-13T09:18:13.288877+0000","last_undegraded":"2024-09-13T09:18:13.288877+0000","last_fullsized":"2024-09-13T09:18:13.288877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:00.757881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042467999999999998,"stat_sum":{"num_bytes":13942805,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":3438,"num_read_kb":6709,"num_write":475,"num_write_kb":1788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":65341,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"61'87","reported_seq":1095,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:18:14.080926+0000","last_change":"2024-09-13T09:18:04.085791+0000","last_active":"2024-09-13T09:18:14.080926+0000","last_peered":"2024-09-13T09:18:14.080926+0000","last_clean":"2024-09-13T09:18:14.080926+0000","last_became_active":"2024-09-13T09:18:03.973063+0000","last_became_peered":"2024-09-13T09:18:03.973063+0000","last_unstale":"2024-09-13T09:18:14.080926+0000","last_undegraded":"2024-09-13T09:18:14.080926+0000","last_fullsized":"2024-09-13T09:18:14.080926+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:29.911994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077588799999999997,"stat_sum":{"num_bytes":8748288,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":800,"num_read_kb":2247,"num_write":85,"num_write_kb":1211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":889520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"61'179","reported_seq":1339,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.060555+0000","last_change":"2024-09-13T09:18:06.977335+0000","last_active":"2024-09-13T09:18:13.060555+0000","last_peered":"2024-09-13T09:18:13.060555+0000","last_clean":"2024-09-13T09:18:13.060555+0000","last_became_active":"2024-09-13T09:18:06.975016+0000","last_became_peered":"2024-09-13T09:18:06.975016+0000","last_unstale":"2024-09-13T09:18:13.060555+0000","last_undegraded":"2024-09-13T09:18:13.060555+0000","last_fullsized":"2024-09-13T09:18:13.060555+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:30.489567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023026689999999998,"stat_sum":{"num_bytes":28851875,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":1423,"num_read_kb":6353,"num_write":218,"num_write_kb":870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":439303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"62'90","reported_seq":860,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.282367+0000","last_change":"2024-09-13T09:17:40.836458+0000","last_active":"2024-09-13T09:18:13.282367+0000","last_peered":"2024-09-13T09:18:13.282367+0000","last_clean":"2024-09-13T09:18:13.282367+0000","last_became_active":"2024-09-13T09:17:40.797607+0000","last_became_peered":"2024-09-13T09:17:40.797607+0000","last_unstale":"2024-09-13T09:18:13.282367+0000","last_undegraded":"2024-09-13T09:18:13.282367+0000","last_fullsized":"2024-09-13T09:18:13.282367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:09.025976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001136335,"stat_sum":{"num_bytes":27537504,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":710,"num_read_kb":2742,"num_write":89,"num_write_kb":918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":42856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"62'167","reported_seq":761,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.521929+0000","last_change":"2024-09-13T09:17:43.811230+0000","last_active":"2024-09-13T09:18:13.521929+0000","last_peered":"2024-09-13T09:18:13.521929+0000","last_clean":"2024-09-13T09:18:13.521929+0000","last_became_active":"2024-09-13T09:17:43.809533+0000","last_became_peered":"2024-09-13T09:17:43.809533+0000","last_unstale":"2024-09-13T09:18:13.521929+0000","last_undegraded":"2024-09-13T09:18:13.521929+0000","last_fullsized":"2024-09-13T09:18:13.521929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:20.389195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001841894,"stat_sum":{"num_bytes":4085620,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":1324,"num_read_kb":1622,"num_write":158,"num_write_kb":955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":16384,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"61'97","reported_seq":553,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.520314+0000","last_change":"2024-09-13T09:17:28.692331+0000","last_active":"2024-09-13T09:18:13.520314+0000","last_peered":"2024-09-13T09:18:13.520314+0000","last_clean":"2024-09-13T09:18:13.520314+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:18:13.520314+0000","last_undegraded":"2024-09-13T09:18:13.520314+0000","last_fullsized":"2024-09-13T09:18:13.520314+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:29.849557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016477340000000001,"stat_sum":{"num_bytes":24184065,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":536,"num_read_kb":2699,"num_write":109,"num_write_kb":1013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"62'87","reported_seq":879,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.180259+0000","last_change":"2024-09-13T09:17:28.684144+0000","last_active":"2024-09-13T09:18:13.180259+0000","last_peered":"2024-09-13T09:18:13.180259+0000","last_clean":"2024-09-13T09:18:13.180259+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:18:13.180259+0000","last_undegraded":"2024-09-13T09:18:13.180259+0000","last_fullsized":"2024-09-13T09:18:13.180259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:10.717628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085639099999999997,"stat_sum":{"num_bytes":12712511,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":769,"num_read_kb":3899,"num_write":85,"num_write_kb":1271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"62'306","reported_seq":453,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.458857+0000","last_change":"2024-09-13T09:17:28.681431+0000","last_active":"2024-09-13T09:18:13.458857+0000","last_peered":"2024-09-13T09:18:13.458857+0000","last_clean":"2024-09-13T09:18:13.458857+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:18:13.458857+0000","last_undegraded":"2024-09-13T09:18:13.458857+0000","last_fullsized":"2024-09-13T09:18:13.458857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:42.341412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083138199999999998,"stat_sum":{"num_bytes":18425046,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":1178,"num_read_kb":2569,"num_write":485,"num_write_kb":7302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.063080+0000","last_change":"2024-09-13T09:17:57.918052+0000","last_active":"2024-09-13T09:18:13.063080+0000","last_peered":"2024-09-13T09:18:13.063080+0000","last_clean":"2024-09-13T09:18:13.063080+0000","last_became_active":"2024-09-13T09:17:57.917112+0000","last_became_peered":"2024-09-13T09:17:57.917112+0000","last_unstale":"2024-09-13T09:18:13.063080+0000","last_undegraded":"2024-09-13T09:18:13.063080+0000","last_fullsized":"2024-09-13T09:18:13.063080+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:44.462499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"62'128","reported_seq":170,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.370781+0000","last_change":"2024-09-13T09:18:04.241645+0000","last_active":"2024-09-13T09:18:13.370781+0000","last_peered":"2024-09-13T09:18:13.370781+0000","last_clean":"2024-09-13T09:18:13.370781+0000","last_became_active":"2024-09-13T09:18:03.979234+0000","last_became_peered":"2024-09-13T09:18:03.979234+0000","last_unstale":"2024-09-13T09:18:13.370781+0000","last_undegraded":"2024-09-13T09:18:13.370781+0000","last_fullsized":"2024-09-13T09:18:13.370781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:42:02.026772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002893765,"stat_sum":{"num_bytes":10672675,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":1183,"num_read_kb":1710,"num_write":126,"num_write_kb":930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":380627,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.060734+0000","last_change":"2024-09-13T09:18:03.969020+0000","last_active":"2024-09-13T09:18:13.060734+0000","last_peered":"2024-09-13T09:18:13.060734+0000","last_clean":"2024-09-13T09:18:13.060734+0000","last_became_active":"2024-09-13T09:18:03.968249+0000","last_became_peered":"2024-09-13T09:18:03.968249+0000","last_unstale":"2024-09-13T09:18:13.060734+0000","last_undegraded":"2024-09-13T09:18:13.060734+0000","last_fullsized":"2024-09-13T09:18:13.060734+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:00.854437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"62'109","reported_seq":888,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.674553+0000","last_change":"2024-09-13T09:17:28.684573+0000","last_active":"2024-09-13T09:18:13.674553+0000","last_peered":"2024-09-13T09:18:13.674553+0000","last_clean":"2024-09-13T09:18:13.674553+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:18:13.674553+0000","last_undegraded":"2024-09-13T09:18:13.674553+0000","last_fullsized":"2024-09-13T09:18:13.674553+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:51.225828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086084099999999999,"stat_sum":{"num_bytes":11421310,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":752,"num_read_kb":2373,"num_write":102,"num_write_kb":942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.062976+0000","last_change":"2024-09-13T09:17:40.794662+0000","last_active":"2024-09-13T09:18:13.062976+0000","last_peered":"2024-09-13T09:18:13.062976+0000","last_clean":"2024-09-13T09:18:13.062976+0000","last_became_active":"2024-09-13T09:17:40.793882+0000","last_became_peered":"2024-09-13T09:17:40.793882+0000","last_unstale":"2024-09-13T09:18:13.062976+0000","last_undegraded":"2024-09-13T09:18:13.062976+0000","last_fullsized":"2024-09-13T09:18:13.062976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:22.922753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"62'108","reported_seq":126,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.733361+0000","last_change":"2024-09-13T09:18:04.163142+0000","last_active":"2024-09-13T09:18:13.733361+0000","last_peered":"2024-09-13T09:18:13.733361+0000","last_clean":"2024-09-13T09:18:13.733361+0000","last_became_active":"2024-09-13T09:18:03.973314+0000","last_became_peered":"2024-09-13T09:18:03.973314+0000","last_unstale":"2024-09-13T09:18:13.733361+0000","last_undegraded":"2024-09-13T09:18:13.733361+0000","last_fullsized":"2024-09-13T09:18:13.733361+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.256457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024125790000000002,"stat_sum":{"num_bytes":22802988,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":241,"num_read_kb":1053,"num_write":98,"num_write_kb":1053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":167476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.060629+0000","last_change":"2024-09-13T09:18:03.968679+0000","last_active":"2024-09-13T09:18:13.060629+0000","last_peered":"2024-09-13T09:18:13.060629+0000","last_clean":"2024-09-13T09:18:13.060629+0000","last_became_active":"2024-09-13T09:18:03.967133+0000","last_became_peered":"2024-09-13T09:18:03.967133+0000","last_unstale":"2024-09-13T09:18:13.060629+0000","last_undegraded":"2024-09-13T09:18:13.060629+0000","last_fullsized":"2024-09-13T09:18:13.060629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:06.643934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"61'96","reported_seq":352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.520213+0000","last_change":"2024-09-13T09:17:28.692726+0000","last_active":"2024-09-13T09:18:13.520213+0000","last_peered":"2024-09-13T09:18:13.520213+0000","last_clean":"2024-09-13T09:18:13.520213+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:18:13.520213+0000","last_undegraded":"2024-09-13T09:18:13.520213+0000","last_fullsized":"2024-09-13T09:18:13.520213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:00.161632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018814960000000001,"stat_sum":{"num_bytes":36148301,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":283,"num_read_kb":3791,"num_write":94,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"62'74","reported_seq":1276,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.782662+0000","last_change":"2024-09-13T09:17:40.875161+0000","last_active":"2024-09-13T09:18:13.782662+0000","last_peered":"2024-09-13T09:18:13.782662+0000","last_clean":"2024-09-13T09:18:13.782662+0000","last_became_active":"2024-09-13T09:17:40.800027+0000","last_became_peered":"2024-09-13T09:17:40.800027+0000","last_unstale":"2024-09-13T09:18:13.782662+0000","last_undegraded":"2024-09-13T09:18:13.782662+0000","last_fullsized":"2024-09-13T09:18:13.782662+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:10.182783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066593499999999999,"stat_sum":{"num_bytes":11912633,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":1146,"num_read_kb":4110,"num_write":69,"num_write_kb":404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":29248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":95,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.053313+0000","last_change":"2024-09-13T09:18:03.976890+0000","last_active":"2024-09-13T09:18:13.053313+0000","last_peered":"2024-09-13T09:18:13.053313+0000","last_clean":"2024-09-13T09:18:13.053313+0000","last_became_active":"2024-09-13T09:18:03.970835+0000","last_became_peered":"2024-09-13T09:18:03.970835+0000","last_unstale":"2024-09-13T09:18:13.053313+0000","last_undegraded":"2024-09-13T09:18:13.053313+0000","last_fullsized":"2024-09-13T09:18:13.053313+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:05.189083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"62'181","reported_seq":703,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.674525+0000","last_change":"2024-09-13T09:18:04.075705+0000","last_active":"2024-09-13T09:18:13.674525+0000","last_peered":"2024-09-13T09:18:13.674525+0000","last_clean":"2024-09-13T09:18:13.674525+0000","last_became_active":"2024-09-13T09:18:03.970479+0000","last_became_peered":"2024-09-13T09:18:03.970479+0000","last_unstale":"2024-09-13T09:18:13.674525+0000","last_undegraded":"2024-09-13T09:18:13.674525+0000","last_fullsized":"2024-09-13T09:18:13.674525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:06.229840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043683300000000002,"stat_sum":{"num_bytes":8178701,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":2517,"num_read_kb":4281,"num_write":181,"num_write_kb":831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1031390,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":56,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:18:12.043048+0000","last_change":"2024-09-13T09:17:19.575553+0000","last_active":"2024-09-13T09:18:12.043048+0000","last_peered":"2024-09-13T09:18:12.043048+0000","last_clean":"2024-09-13T09:18:12.043048+0000","last_became_active":"2024-09-13T09:17:17.493930+0000","last_became_peered":"2024-09-13T09:17:17.493930+0000","last_unstale":"2024-09-13T09:18:12.043048+0000","last_undegraded":"2024-09-13T09:18:12.043048+0000","last_fullsized":"2024-09-13T09:18:12.043048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:12.092799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000590198,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"61'104","reported_seq":894,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.281582+0000","last_change":"2024-09-13T09:17:40.870470+0000","last_active":"2024-09-13T09:18:13.281582+0000","last_peered":"2024-09-13T09:18:13.281582+0000","last_clean":"2024-09-13T09:18:13.281582+0000","last_became_active":"2024-09-13T09:17:40.793065+0000","last_became_peered":"2024-09-13T09:17:40.793065+0000","last_unstale":"2024-09-13T09:18:13.281582+0000","last_undegraded":"2024-09-13T09:18:13.281582+0000","last_fullsized":"2024-09-13T09:18:13.281582+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:38.586460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052255699999999999,"stat_sum":{"num_bytes":13235581,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":809,"num_read_kb":10955,"num_write":105,"num_write_kb":4986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.062759+0000","last_change":"2024-09-13T09:18:03.968508+0000","last_active":"2024-09-13T09:18:13.062759+0000","last_peered":"2024-09-13T09:18:13.062759+0000","last_clean":"2024-09-13T09:18:13.062759+0000","last_became_active":"2024-09-13T09:18:03.968092+0000","last_became_peered":"2024-09-13T09:18:03.968092+0000","last_unstale":"2024-09-13T09:18:13.062759+0000","last_undegraded":"2024-09-13T09:18:13.062759+0000","last_fullsized":"2024-09-13T09:18:13.062759+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:39.649827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"62'123","reported_seq":999,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.179637+0000","last_change":"2024-09-13T09:17:28.686727+0000","last_active":"2024-09-13T09:18:13.179637+0000","last_peered":"2024-09-13T09:18:13.179637+0000","last_clean":"2024-09-13T09:18:13.179637+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:18:13.179637+0000","last_undegraded":"2024-09-13T09:18:13.179637+0000","last_fullsized":"2024-09-13T09:18:13.179637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:44.094850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061431400000000003,"stat_sum":{"num_bytes":35529168,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":883,"num_read_kb":4408,"num_write":146,"num_write_kb":6730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":58,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.053255+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:18:13.053255+0000","last_peered":"2024-09-13T09:18:13.053255+0000","last_clean":"2024-09-13T09:18:13.053255+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:18:13.053255+0000","last_undegraded":"2024-09-13T09:18:13.053255+0000","last_fullsized":"2024-09-13T09:18:13.053255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.062818+0000","last_change":"2024-09-13T09:17:43.803525+0000","last_active":"2024-09-13T09:18:13.062818+0000","last_peered":"2024-09-13T09:18:13.062818+0000","last_clean":"2024-09-13T09:18:13.062818+0000","last_became_active":"2024-09-13T09:17:43.802603+0000","last_became_peered":"2024-09-13T09:17:43.802603+0000","last_unstale":"2024-09-13T09:18:13.062818+0000","last_undegraded":"2024-09-13T09:18:13.062818+0000","last_fullsized":"2024-09-13T09:18:13.062818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:07.561959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"63'284","reported_seq":758,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:18:14.052766+0000","last_change":"2024-09-13T09:18:04.153662+0000","last_active":"2024-09-13T09:18:14.052766+0000","last_peered":"2024-09-13T09:18:14.052766+0000","last_clean":"2024-09-13T09:18:14.052766+0000","last_became_active":"2024-09-13T09:18:03.974569+0000","last_became_peered":"2024-09-13T09:18:03.974569+0000","last_unstale":"2024-09-13T09:18:14.052766+0000","last_undegraded":"2024-09-13T09:18:14.052766+0000","last_fullsized":"2024-09-13T09:18:14.052766+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:43.479751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014243249999999999,"stat_sum":{"num_bytes":29802270,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":3,"num_read":1644,"num_read_kb":4072,"num_write":408,"num_write_kb":2210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":741496,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"62'296","reported_seq":718,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.761753+0000","last_change":"2024-09-13T09:17:28.681630+0000","last_active":"2024-09-13T09:18:13.761753+0000","last_peered":"2024-09-13T09:18:13.761753+0000","last_clean":"2024-09-13T09:18:13.761753+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:18:13.761753+0000","last_undegraded":"2024-09-13T09:18:13.761753+0000","last_fullsized":"2024-09-13T09:18:13.761753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:06.071033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013508260000000001,"stat_sum":{"num_bytes":5067962,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":1734,"num_read_kb":1918,"num_write":379,"num_write_kb":860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"62'190","reported_seq":1264,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.679698+0000","last_change":"2024-09-13T09:18:06.978086+0000","last_active":"2024-09-13T09:18:13.679698+0000","last_peered":"2024-09-13T09:18:13.679698+0000","last_clean":"2024-09-13T09:18:13.679698+0000","last_became_active":"2024-09-13T09:18:06.976785+0000","last_became_peered":"2024-09-13T09:18:06.976785+0000","last_unstale":"2024-09-13T09:18:13.679698+0000","last_undegraded":"2024-09-13T09:18:13.679698+0000","last_fullsized":"2024-09-13T09:18:13.679698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:19.887646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033158900000000003,"stat_sum":{"num_bytes":11788846,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":1828,"num_read_kb":3991,"num_write":181,"num_write_kb":1188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":387969,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"62'214","reported_seq":499,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.674997+0000","last_change":"2024-09-13T09:17:40.896216+0000","last_active":"2024-09-13T09:18:13.674997+0000","last_peered":"2024-09-13T09:18:13.674997+0000","last_clean":"2024-09-13T09:18:13.674997+0000","last_became_active":"2024-09-13T09:17:40.801195+0000","last_became_peered":"2024-09-13T09:17:40.801195+0000","last_unstale":"2024-09-13T09:18:13.674997+0000","last_undegraded":"2024-09-13T09:18:13.674997+0000","last_fullsized":"2024-09-13T09:18:13.674997+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:00:07.773428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002522725,"stat_sum":{"num_bytes":19534893,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":1015,"num_read_kb":2195,"num_write":305,"num_write_kb":7509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67680,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"62'311","reported_seq":386,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.293071+0000","last_change":"2024-09-13T09:17:28.693070+0000","last_active":"2024-09-13T09:18:13.293071+0000","last_peered":"2024-09-13T09:18:13.293071+0000","last_clean":"2024-09-13T09:18:13.293071+0000","last_became_active":"2024-09-13T09:17:26.698657+0000","last_became_peered":"2024-09-13T09:17:26.698657+0000","last_unstale":"2024-09-13T09:18:13.293071+0000","last_undegraded":"2024-09-13T09:18:13.293071+0000","last_fullsized":"2024-09-13T09:18:13.293071+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:12.388714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002213343,"stat_sum":{"num_bytes":5146854,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":967,"num_read_kb":2066,"num_write":446,"num_write_kb":2568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"62'133","reported_seq":367,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.292367+0000","last_change":"2024-09-13T09:17:28.684477+0000","last_active":"2024-09-13T09:18:13.292367+0000","last_peered":"2024-09-13T09:18:13.292367+0000","last_clean":"2024-09-13T09:18:13.292367+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:18:13.292367+0000","last_undegraded":"2024-09-13T09:18:13.292367+0000","last_fullsized":"2024-09-13T09:18:13.292367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:25.411265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099816899999999992,"stat_sum":{"num_bytes":28558438,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":231,"num_read_kb":1756,"num_write":156,"num_write_kb":7903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"62'186","reported_seq":358,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.179264+0000","last_change":"2024-09-13T09:17:28.687062+0000","last_active":"2024-09-13T09:18:13.179264+0000","last_peered":"2024-09-13T09:18:13.179264+0000","last_clean":"2024-09-13T09:18:13.179264+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:18:13.179264+0000","last_undegraded":"2024-09-13T09:18:13.179264+0000","last_fullsized":"2024-09-13T09:18:13.179264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:31.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034733699999999999,"stat_sum":{"num_bytes":15398067,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":639,"num_read_kb":1993,"num_write":252,"num_write_kb":1359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"62'735","reported_seq":1575,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.276278+0000","last_change":"2024-09-13T09:17:28.684481+0000","last_active":"2024-09-13T09:18:13.276278+0000","last_peered":"2024-09-13T09:18:13.276278+0000","last_clean":"2024-09-13T09:18:13.276278+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:18:13.276278+0000","last_undegraded":"2024-09-13T09:18:13.276278+0000","last_fullsized":"2024-09-13T09:18:13.276278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:56.423075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151104,"stat_sum":{"num_bytes":12578626,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":4199,"num_read_kb":5638,"num_write":1229,"num_write_kb":1658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"62'108","reported_seq":296,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.521703+0000","last_change":"2024-09-13T09:17:57.986034+0000","last_active":"2024-09-13T09:18:13.521703+0000","last_peered":"2024-09-13T09:18:13.521703+0000","last_clean":"2024-09-13T09:18:13.521703+0000","last_became_active":"2024-09-13T09:17:57.919132+0000","last_became_peered":"2024-09-13T09:17:57.919132+0000","last_unstale":"2024-09-13T09:18:13.521703+0000","last_undegraded":"2024-09-13T09:18:13.521703+0000","last_fullsized":"2024-09-13T09:18:13.521703+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:49.597931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037557999999999998,"stat_sum":{"num_bytes":3588316,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":216,"num_read_kb":455,"num_write":123,"num_write_kb":778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":20268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"62'91","reported_seq":594,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.724975+0000","last_change":"2024-09-13T09:17:28.683788+0000","last_active":"2024-09-13T09:18:13.724975+0000","last_peered":"2024-09-13T09:18:13.724975+0000","last_clean":"2024-09-13T09:18:13.724975+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:18:13.724975+0000","last_undegraded":"2024-09-13T09:18:13.724975+0000","last_fullsized":"2024-09-13T09:18:13.724975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:47.572839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070075400000000005,"stat_sum":{"num_bytes":6598909,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":476,"num_read_kb":3271,"num_write":89,"num_write_kb":1343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"63'193","reported_seq":1218,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:18:14.082795+0000","last_change":"2024-09-13T09:17:40.816999+0000","last_active":"2024-09-13T09:18:14.082795+0000","last_peered":"2024-09-13T09:18:14.082795+0000","last_clean":"2024-09-13T09:18:14.082795+0000","last_became_active":"2024-09-13T09:17:40.789499+0000","last_became_peered":"2024-09-13T09:17:40.789499+0000","last_unstale":"2024-09-13T09:18:14.082795+0000","last_undegraded":"2024-09-13T09:18:14.082795+0000","last_fullsized":"2024-09-13T09:18:14.082795+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:13.395268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084252600000000002,"stat_sum":{"num_bytes":23315181,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2062,"num_read_kb":4921,"num_write":200,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1322,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"62'84","reported_seq":891,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:18:13.768042+0000","last_change":"2024-09-13T09:17:28.683729+0000","last_active":"2024-09-13T09:18:13.768042+0000","last_peered":"2024-09-13T09:18:13.768042+0000","last_clean":"2024-09-13T09:18:13.768042+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:18:13.768042+0000","last_undegraded":"2024-09-13T09:18:13.768042+0000","last_fullsized":"2024-09-13T09:18:13.768042+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:07.893848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000594467,"stat_sum":{"num_bytes":12127016,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":794,"num_read_kb":3562,"num_write":79,"num_write_kb":926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":520170916,"num_objects":561,"num_object_clones":99,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":135,"num_read":41233,"num_read_kb":108640,"num_write":7824,"num_write_kb":85996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":577,"num_bytes_recovered":4820954,"num_keys_recovered":127,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19865600,"data_stored":115547127,"data_compressed":1456179,"data_compressed_allocated":7749632,"data_compressed_original":105436591,"omap_allocated":0,"internal_metadata":0},"log_size":6072,"ondisk_log_size":6072,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":30095,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42764,"kb_used_data":3452,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94329076,"statfs":{"total":96636764160,"available":96592973824,"internally_reserved":0,"allocated":3534848,"data_stored":17879189,"data_compressed":168901,"data_compressed_allocated":1150976,"data_compressed_original":16203776,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.2360000000000002,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.873}]},{"osd":1,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.92700000000000005},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":3.9039999999999999,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.81200000000000006}]},{"osd":3,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.94799999999999995}]},{"osd":4,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":3.6139999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.625}]},{"osd":5,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.125,"5min":3.125,"15min":3.125},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.3130000000000002,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.68100000000000005}]}]},{"osd":6,"up_from":0,"seq":55834574864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44180,"kb_used_data":4844,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327660,"statfs":{"total":96636764160,"available":96591523840,"internally_reserved":0,"allocated":4960256,"data_stored":31922981,"data_compressed":315680,"data_compressed_allocated":2048000,"data_compressed_original":29736960,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":7.8150000000000004,"5min":7.8150000000000004,"15min":7.8150000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":132.32900000000001,"5min":132.32900000000001,"15min":132.32900000000001},"last":1.091},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.236}]},{"osd":1,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.4419999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":8.2629999999999999,"5min":8.2629999999999999,"15min":8.2629999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":132.23599999999999,"5min":132.23599999999999,"15min":132.23599999999999},"last":1.284}]},{"osd":6,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.0569999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.383},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.5910000000000002,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.6379999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31000,"kb_used_data":3976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340840,"statfs":{"total":96636764160,"available":96605020160,"internally_reserved":0,"allocated":4071424,"data_stored":21224840,"data_compressed":299835,"data_compressed_allocated":1462272,"data_compressed_original":19320832,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.77600000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.95199999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.331}]},{"osd":5,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.92300000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.165},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.2789999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.2470000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43216,"kb_used_data":3912,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94328624,"statfs":{"total":96636764160,"available":96592510976,"internally_reserved":0,"allocated":4005888,"data_stored":19581105,"data_compressed":240462,"data_compressed_allocated":1269760,"data_compressed_original":17603621,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":1.5720000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.282},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.89500000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.544},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.68100000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.669},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.1619999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.22}]},{"osd":6,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.347},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":0.96699999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":15.4,"5min":15.4,"15min":15.4},"last":1.47}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36724,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335116,"statfs":{"total":96636764160,"available":96599158784,"internally_reserved":0,"allocated":1527808,"data_stored":1718650,"data_compressed":33249,"data_compressed_allocated":86016,"data_compressed_original":802816,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.67600000000000005}]},{"osd":3,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.78800000000000003}]},{"osd":5,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.9809999999999999,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.85399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37536,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334304,"statfs":{"total":96636764160,"available":96598327296,"internally_reserved":0,"allocated":2379776,"data_stored":4044667,"data_compressed":98758,"data_compressed_allocated":286720,"data_compressed_original":2632074,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.47199999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.42699999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":8.3460000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.71099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39260,"kb_used_data":4020,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332580,"statfs":{"total":96636764160,"available":96596561920,"internally_reserved":0,"allocated":4116480,"data_stored":21839273,"data_compressed":287339,"data_compressed_allocated":1454080,"data_compressed_original":19824640,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.40699999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.58099999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.623},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.246}]},{"osd":5,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.151},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.111}]},{"osd":6,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.018},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.82499999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":0.95699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3358720,"data_stored":20929682,"data_compressed":276961,"data_compressed_allocated":1417216,"data_compressed_original":19234816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1642496,"data_stored":3718522,"data_compressed":98758,"data_compressed_allocated":286720,"data_compressed_original":2632074,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":901120,"data_stored":1501575,"data_compressed":33396,"data_compressed_allocated":90112,"data_compressed_original":831488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3284992,"data_stored":19261127,"data_compressed":240462,"data_compressed_allocated":1269760,"data_compressed_original":17603621,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3354624,"data_stored":20904946,"data_compressed":299835,"data_compressed_allocated":1462272,"data_compressed_original":19320832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4280320,"data_stored":31629138,"data_compressed":315680,"data_compressed_allocated":2048000,"data_compressed_original":29736960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":609108,"data_compressed":31525,"data_compressed_allocated":61440,"data_compressed_original":446464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2785280,"data_stored":16993029,"data_compressed":159562,"data_compressed_allocated":1114112,"data_compressed_original":15630336,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:18:15.847 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:18:15.847 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:18:15.848 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-13T09:18:15.848 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:18:15.848 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:18:15.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-13T09:18:15.890 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:15.891+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:15.925 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:15.925 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:15.925 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:15.925 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:15.940 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:15.982 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:15.983+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:15.985 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:15.986 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:18:15.986 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:16.068 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:16.067+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:16.110 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:16.110 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:16.110 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:16.110 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:16.110 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:16.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:16.169 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:16.162+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:16.229 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone18 from 0xc804fd3 (overlap 0x7198b99) to 0x221437a 2024-09-13T09:18:16.270 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:16.266+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:16.323 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.786819433000002, 2024-09-13T09:18:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:18:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:18:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:18:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:18:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:18:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:18:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:18:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:18:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:18:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:18:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:18:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19325 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2514944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393342767, 2024-09-13T09:18:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041428072000000003, 2024-09-13T09:18:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19325 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929837+0000", 2024-09-13T09:18:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929839+0000", 2024-09-13T09:18:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:18:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929845+0000", 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9510000000000003e-06 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929850+0000", 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971169+0000", 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041318743999999998 2024-09-13T09:18:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971214+0000", 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971265+0000", 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0963e-05 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19326 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2613248~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393325830999999, 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041534459000000003, 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19326 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929854+0000", 2024-09-13T09:18:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929856+0000", 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0470000000000001e-06 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929862+0000", 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3300000000000004e-06 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929867+0000", 2024-09-13T09:18:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:18:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971293+0000", 2024-09-13T09:18:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041425785999999999 2024-09-13T09:18:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971337+0000", 2024-09-13T09:18:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4240999999999999e-05 2024-09-13T09:18:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971389+0000", 2024-09-13T09:18:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1483999999999997e-05 2024-09-13T09:18:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19327 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2711552~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393308765, 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041642875000000003, 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19327 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929871+0000", 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929873+0000", 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.26e-06 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929879+0000", 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0519999999999998e-06 2024-09-13T09:18:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929884+0000", 2024-09-13T09:18:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:18:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971416+0000", 2024-09-13T09:18:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041532122999999997 2024-09-13T09:18:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971460+0000", 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4456999999999998e-05 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971514+0000", 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3579000000000003e-05 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19328 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2809856~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393292001999999, 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041750662000000001, 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19328 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929888+0000", 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929890+0000", 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929896+0000", 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929901+0000", 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971542+0000", 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041640442 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971587+0000", 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4972999999999998e-05 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971639+0000", 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1990000000000002e-05 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19329 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 2908160~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393250009999999, 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041833958999999997, 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19329 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929930+0000", 2024-09-13T09:18:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929929+0000", 2024-09-13T09:18:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929932+0000", 2024-09-13T09:18:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0699999999999998e-06 2024-09-13T09:18:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929941+0000", 2024-09-13T09:18:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.149e-06 2024-09-13T09:18:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929947+0000", 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.699e-06 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971666+0000", 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041719224999999999 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971711+0000", 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4823000000000001e-05 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971764+0000", 2024-09-13T09:18:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2741999999999997e-05 2024-09-13T09:18:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19330 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 3006464~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:18:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393227269, 2024-09-13T09:18:16.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041949980999999997, 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19330 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929953+0000", 2024-09-13T09:18:16.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929952+0000", 2024-09-13T09:18:16.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929955+0000", 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929961+0000", 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1199999999999999e-06 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929966+0000", 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971792+0000", 2024-09-13T09:18:16.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041826680999999998 2024-09-13T09:18:16.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971838+0000", 2024-09-13T09:18:16.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5939000000000002e-05 2024-09-13T09:18:16.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971903+0000", 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4499999999999996e-05 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19331 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 3104768~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393210302, 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042058848000000003, 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19331 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929970+0000", 2024-09-13T09:18:16.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929969+0000", 2024-09-13T09:18:16.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929975+0000", 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.434e-06 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929981+0000", 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929986+0000", 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1780000000000002e-06 2024-09-13T09:18:16.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971931+0000", 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041944417999999997 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.971976+0000", 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4962000000000001e-05 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972029+0000", 2024-09-13T09:18:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2768e-05 2024-09-13T09:18:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19332 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 65536~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:18:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393183132000001, 2024-09-13T09:18:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042158568, 2024-09-13T09:18:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19332 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929997+0000", 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929996+0000", 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.929999+0000", 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.306e-06 2024-09-13T09:18:16.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930005+0000", 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0379999999999996e-06 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930010+0000", 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0969999999999996e-06 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972057+0000", 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042047117000000002 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972102+0000", 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5422e-05 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972155+0000", 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2964999999999999e-05 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19333 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 163840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393166042000001, 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042266805999999997, 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19333 2024-09-13T09:18:16.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.371 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:16.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:16.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930014+0000", 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930016+0000", 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930022+0000", 2024-09-13T09:18:16.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2550000000000003e-06 2024-09-13T09:18:16.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930027+0000", 2024-09-13T09:18:16.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8060000000000002e-06 2024-09-13T09:18:16.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972183+0000", 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042156276999999999 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972228+0000", 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5317e-05 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972281+0000", 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2398999999999998e-05 2024-09-13T09:18:16.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19334 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 262144~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393148196999999, 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042375573, 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19334 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930032+0000", 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930031+0000", 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930034+0000", 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930040+0000", 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0789999999999997e-06 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930045+0000", 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0139999999999998e-06 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972307+0000", 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042262685000000001 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972352+0000", 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5021000000000001e-05 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972407+0000", 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4945000000000003e-05 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19335 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 360448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393131383, 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042483785000000003, 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19335 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930049+0000", 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930048+0000", 2024-09-13T09:18:16.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930050+0000", 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.136e-06 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930056+0000", 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930061+0000", 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9289999999999997e-06 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972435+0000", 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042373347999999998 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972479+0000", 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4597e-05 2024-09-13T09:18:16.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972532+0000", 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3034000000000001e-05 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19336 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 458752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393114445999998, 2024-09-13T09:18:16.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042590488000000003, 2024-09-13T09:18:16.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.385 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19336 2024-09-13T09:18:16.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.385 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:18:16.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930066+0000", 2024-09-13T09:18:16.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930065+0000", 2024-09-13T09:18:16.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930067+0000", 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2110000000000001e-06 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930074+0000", 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4640000000000003e-06 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930079+0000", 2024-09-13T09:18:16.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7709999999999997e-06 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972560+0000", 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042481738999999998 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972604+0000", 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4060999999999997e-05 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972656+0000", 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.164e-05 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19337 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.393097270999998, 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042697997000000001, 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19337 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930083+0000", 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930082+0000", 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930085+0000", 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.221e-06 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930091+0000", 2024-09-13T09:18:16.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.942e-06 2024-09-13T09:18:16.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.930096+0000", 2024-09-13T09:18:16.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.04e-06 2024-09-13T09:18:16.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972683+0000", 2024-09-13T09:18:16.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042587885999999998 2024-09-13T09:18:16.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972728+0000", 2024-09-13T09:18:16.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4058999999999999e-05 2024-09-13T09:18:16.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.972781+0000", 2024-09-13T09:18:16.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3156999999999999e-05 2024-09-13T09:18:16.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.393 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.394 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19342 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1048576~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.394 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:16.394 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.391179844, 2024-09-13T09:18:16.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041418260999999998, 2024-09-13T09:18:16.394 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.394 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.394 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19342 2024-09-13T09:18:16.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.395 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:16.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:16.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932000+0000", 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932002+0000", 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3180000000000001e-06 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932008+0000", 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2199999999999997e-06 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932013+0000", 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973319+0000", 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041305738000000002 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973364+0000", 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5809000000000001e-05 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973418+0000", 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3888000000000003e-05 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19343 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1146880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.391152637000001, 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041515107000000002, 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19343 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932027+0000", 2024-09-13T09:18:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932034+0000", 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932040+0000", 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2260000000000002e-06 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932045+0000", 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5589999999999999e-06 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973446+0000", 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041401070999999998 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973491+0000", 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5021999999999997e-05 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973542+0000", 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1465000000000003e-05 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19344 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.391130110999999, 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041617848999999998, 2024-09-13T09:18:16.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19344 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932050+0000", 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932049+0000", 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932052+0000", 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4540000000000001e-06 2024-09-13T09:18:16.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932064+0000", 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1911e-05 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932079+0000", 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4765e-05 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973570+0000", 2024-09-13T09:18:16.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041491800000000002 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973614+0000", 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4149e-05 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973668+0000", 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3198999999999997e-05 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19345 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1343488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.391093940000001, 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041707699000000001, 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19345 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.408 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932086+0000", 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932085+0000", 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932089+0000", 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7009999999999998e-06 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932097+0000", 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3010000000000007e-06 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932104+0000", 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9260000000000003e-06 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973695+0000", 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041591056000000001 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973741+0000", 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6003e-05 2024-09-13T09:18:16.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973794+0000", 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2265999999999997e-05 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19346 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1441792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.391069839, 2024-09-13T09:18:16.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041808933999999999, 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19346 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932110+0000", 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932112+0000", 2024-09-13T09:18:16.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:18:16.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932119+0000", 2024-09-13T09:18:16.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8369999999999999e-06 2024-09-13T09:18:16.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932125+0000", 2024-09-13T09:18:16.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:16.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973822+0000", 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041697369999999997 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973867+0000", 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4648999999999999e-05 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973919+0000", 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2451000000000003e-05 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19347 3.19 3:99c177c3:::rbd_data.1185c2de398f.0000000000000006:e [read 1540096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.391050386, 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041915045999999997, 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:16.417 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19347 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932130+0000", 2024-09-13T09:18:16.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932129+0000", 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932131+0000", 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.278e-06 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:16.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932139+0000", 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.43e-06 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.932145+0000", 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7939999999999999e-06 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973946+0000", 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041801724999999998 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.973993+0000", 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6245000000000002e-05 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.974045+0000", 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1950999999999998e-05 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:16.421 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:16.422 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:16.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:16.424 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:16.418+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:16.450 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-13T09:18:16.453 INFO:teuthology.orchestra.run.smithi032.stdout:2806 writesame 0x107ffc1 thru 0x108d118 data_size 0x58(0xd158 bytes) 2024-09-13T09:18:16.498 INFO:teuthology.orchestra.run.smithi032.stdout:2807 read 0x5ba17b5 thru 0x5baec71 (0xd4bd bytes) 2024-09-13T09:18:16.498 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:16.498 INFO:teuthology.orchestra.run.smithi032.stdout:2809 writesame 0xa2174d2 thru 0xa21c1a2 data_size 0x5f(0x4cd1 bytes) 2024-09-13T09:18:16.523 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:16.518+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:16.569 INFO:teuthology.orchestra.run.smithi032.stdout:2811 read 0x47df192 thru 0x47e3ec5 (0x4d34 bytes) 2024-09-13T09:18:16.569 INFO:teuthology.orchestra.run.smithi032.stdout:2812 write 0x8ad8eff thru 0x8adcc9c (0x3d9e bytes) 2024-09-13T09:18:16.589 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28769 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.580954+0000", 2024-09-13T09:18:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.005088029, 2024-09-13T09:18:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051059419999999996, 2024-09-13T09:18:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28769 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580954+0000", 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580954+0000", 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580953+0000", 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580958+0000", 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0690000000000001e-06 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580975+0000", 2024-09-13T09:18:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6345e-05 2024-09-13T09:18:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580988+0000", 2024-09-13T09:18:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2965999999999999e-05 2024-09-13T09:18:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28770 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.580999+0000", 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0050436860000000003, 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051217240000000002, 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28770 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580999+0000", 2024-09-13T09:18:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580999+0000", 2024-09-13T09:18:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580998+0000", 2024-09-13T09:18:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581003+0000", 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581019+0000", 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5523e-05 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581031+0000", 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2257e-05 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28771 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581063+0000", 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0049795070000000002, 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051002840000000001, 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28771 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581063+0000", 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581063+0000", 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581062+0000", 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581067+0000", 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5459999999999999e-06 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581085+0000", 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7591000000000001e-05 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581098+0000", 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.34e-05 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28772 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581109+0000", 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0049328729999999999, 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051138190000000004, 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28772 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581109+0000", 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581109+0000", 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581109+0000", 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581113+0000", 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7509999999999999e-06 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581131+0000", 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7306e-05 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581148+0000", 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7608000000000001e-05 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28773 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581181+0000", 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0048608389999999996, 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050804709999999996, 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28773 2024-09-13T09:18:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581181+0000", 2024-09-13T09:18:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581181+0000", 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581180+0000", 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581186+0000", 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0430000000000004e-06 2024-09-13T09:18:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581204+0000", 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7836999999999999e-05 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581220+0000", 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5406000000000001e-05 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28774 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581232+0000", 2024-09-13T09:18:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0048106080000000001, 2024-09-13T09:18:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050889250000000002, 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28774 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581232+0000", 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581232+0000", 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581231+0000", 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581236+0000", 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2089999999999996e-06 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581252+0000", 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.562e-05 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581264+0000", 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2551000000000001e-05 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28775 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581275+0000", 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0047670029999999997, 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051054899999999999, 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28775 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581275+0000", 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581275+0000", 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581274+0000", 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581279+0000", 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1619999999999999e-06 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581295+0000", 2024-09-13T09:18:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5988999999999999e-05 2024-09-13T09:18:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581307+0000", 2024-09-13T09:18:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1596999999999999e-05 2024-09-13T09:18:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28776 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581319+0000", 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0047236149999999996, 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051221890000000001, 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28776 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581319+0000", 2024-09-13T09:18:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581319+0000", 2024-09-13T09:18:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581318+0000", 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581323+0000", 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9679999999999997e-06 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581339+0000", 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6563e-05 2024-09-13T09:18:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581353+0000", 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3286e-05 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28777 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581363+0000", 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0046794920000000004, 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051368109999999998, 2024-09-13T09:18:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28777 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581363+0000", 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581363+0000", 2024-09-13T09:18:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581362+0000", 2024-09-13T09:18:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581367+0000", 2024-09-13T09:18:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2429999999999996e-06 2024-09-13T09:18:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581383+0000", 2024-09-13T09:18:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5529e-05 2024-09-13T09:18:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581394+0000", 2024-09-13T09:18:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1548e-05 2024-09-13T09:18:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28778 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581435+0000", 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0046074130000000003, 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051019129999999996, 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28778 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581435+0000", 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581435+0000", 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581434+0000", 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581440+0000", 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9150000000000001e-06 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581475+0000", 2024-09-13T09:18:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4947999999999998e-05 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581497+0000", 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2883000000000001e-05 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28779 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581510+0000", 2024-09-13T09:18:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0045319999999999996, 2024-09-13T09:18:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050625009999999996, 2024-09-13T09:18:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28779 2024-09-13T09:18:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581510+0000", 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581510+0000", 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581509+0000", 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581515+0000", 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581534+0000", 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8671000000000001e-05 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581546+0000", 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2731000000000001e-05 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28780 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581557+0000", 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0044852030000000001, 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050789329999999999, 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28780 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581557+0000", 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581557+0000", 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581556+0000", 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581561+0000", 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7770000000000002e-06 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581577+0000", 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5656999999999999e-05 2024-09-13T09:18:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581589+0000", 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2206e-05 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28781 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581599+0000", 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0044429029999999998, 2024-09-13T09:18:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050963850000000002, 2024-09-13T09:18:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28781 2024-09-13T09:18:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581599+0000", 2024-09-13T09:18:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581599+0000", 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581598+0000", 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581604+0000", 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3489999999999999e-06 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581620+0000", 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5899000000000001e-05 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581632+0000", 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2425999999999999e-05 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28782 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581643+0000", 2024-09-13T09:18:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.004399104, 2024-09-13T09:18:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051116929999999996, 2024-09-13T09:18:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28782 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581643+0000", 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581643+0000", 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581642+0000", 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581647+0000", 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0989999999999998e-06 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581663+0000", 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5800000000000001e-05 2024-09-13T09:18:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581675+0000", 2024-09-13T09:18:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1946e-05 2024-09-13T09:18:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28783 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581686+0000", 2024-09-13T09:18:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0043561729999999996, 2024-09-13T09:18:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051173599999999996, 2024-09-13T09:18:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28783 2024-09-13T09:18:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581686+0000", 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581686+0000", 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581685+0000", 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581690+0000", 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8999999999999997e-06 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581706+0000", 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581724+0000", 2024-09-13T09:18:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8614000000000002e-05 2024-09-13T09:18:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28784 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581739+0000", 2024-09-13T09:18:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0043030589999999997, 2024-09-13T09:18:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050950969999999998, 2024-09-13T09:18:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28784 2024-09-13T09:18:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581739+0000", 2024-09-13T09:18:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581739+0000", 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581738+0000", 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581743+0000", 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:18:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581759+0000", 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6116999999999998e-05 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581772+0000", 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2617e-05 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28785 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581821+0000", 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0042212600000000001, 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050502840000000004, 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28785 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581821+0000", 2024-09-13T09:18:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581821+0000", 2024-09-13T09:18:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581820+0000", 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581847+0000", 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7396999999999999e-05 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581867+0000", 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9791000000000001e-05 2024-09-13T09:18:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581887+0000", 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9848999999999999e-05 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28626 3.3 3:c3366d27:::rbd_data.1185833655de.0000000000000040:16 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.576148+0000", 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0098943669999999994, 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01075888, 2024-09-13T09:18:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28626 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576148+0000", 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576148+0000", 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576147+0000", 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576150+0000", 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576171+0000", 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0200999999999999e-05 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576178+0000", 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2899999999999997e-06 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586810+0000", 2024-09-13T09:18:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010632524000000001 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586878+0000", 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7879999999999994e-05 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28786 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581900+0000", 2024-09-13T09:18:16.655 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0041424260000000003, 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050583319999999996, 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28786 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581900+0000", 2024-09-13T09:18:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581900+0000", 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581899+0000", 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581904+0000", 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5110000000000003e-06 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581920+0000", 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5928999999999999e-05 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581932+0000", 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1754e-05 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28627 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.576190+0000", 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0098522209999999995, 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010821704, 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28627 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576190+0000", 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576190+0000", 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576189+0000", 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576193+0000", 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9740000000000004e-06 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576204+0000", 2024-09-13T09:18:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0791000000000001e-05 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576212+0000", 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0269999999999995e-06 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586995+0000", 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010782283 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28787 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581943+0000", 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0040991780000000002, 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051384480000000003, 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28787 2024-09-13T09:18:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581943+0000", 2024-09-13T09:18:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581943+0000", 2024-09-13T09:18:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581942+0000", 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581947+0000", 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2000000000000002e-06 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581963+0000", 2024-09-13T09:18:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5994e-05 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581982+0000", 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8751000000000001e-05 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28628 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.576219+0000", 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0098236599999999997, 2024-09-13T09:18:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010922978, 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28628 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576219+0000", 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576219+0000", 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576218+0000", 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576221+0000", 2024-09-13T09:18:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9840000000000001e-06 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576232+0000", 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1358999999999999e-05 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576240+0000", 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4120000000000004e-06 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28788 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.581993+0000", 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0040487980000000002, 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052066320000000001, 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28788 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581993+0000", 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581993+0000", 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581993+0000", 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.581998+0000", 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1080000000000001e-06 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582013+0000", 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5702999999999999e-05 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582026+0000", 2024-09-13T09:18:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2165e-05 2024-09-13T09:18:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28629 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.576246+0000", 2024-09-13T09:18:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0097965080000000006, 2024-09-13T09:18:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010985382, 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28629 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576246+0000", 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576246+0000", 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576245+0000", 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576248+0000", 2024-09-13T09:18:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8049999999999998e-06 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576258+0000", 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0207000000000001e-05 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576265+0000", 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0929999999999997e-06 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28789 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.582037+0000", 2024-09-13T09:18:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0040053959999999996, 2024-09-13T09:18:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005227939, 2024-09-13T09:18:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28789 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582037+0000", 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582037+0000", 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582036+0000", 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582041+0000", 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0250000000000002e-06 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582056+0000", 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5506e-05 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582068+0000", 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2003e-05 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28630 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.576271+0000", 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0097709700000000003, 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011034924999999999, 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28630 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576271+0000", 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576271+0000", 2024-09-13T09:18:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576271+0000", 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576274+0000", 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.88e-06 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576282+0000", 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1170000000000004e-06 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.576288+0000", 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3090000000000001e-06 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28790 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.582079+0000", 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0039634819999999999, 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052608500000000001, 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28790 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582079+0000", 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582079+0000", 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582078+0000", 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582083+0000", 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7879999999999997e-06 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582107+0000", 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4223000000000002e-05 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.582120+0000", 2024-09-13T09:18:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2763e-05 2024-09-13T09:18:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28759 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.580359+0000", 2024-09-13T09:18:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0056828139999999996, 2024-09-13T09:18:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0070101640000000002, 2024-09-13T09:18:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28759 2024-09-13T09:18:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580359+0000", 2024-09-13T09:18:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580359+0000", 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580358+0000", 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580370+0000", 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2473999999999999e-05 2024-09-13T09:18:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580397+0000", 2024-09-13T09:18:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6452e-05 2024-09-13T09:18:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580415+0000", 2024-09-13T09:18:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8037e-05 2024-09-13T09:18:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28919 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.585674+0000", 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.00036832500000000002, 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017273690000000001, 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28919 2024-09-13T09:18:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585674+0000", 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585674+0000", 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585672+0000", 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585683+0000", 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1683e-05 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585708+0000", 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5211e-05 2024-09-13T09:18:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585726+0000", 2024-09-13T09:18:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7190999999999999e-05 2024-09-13T09:18:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28760 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.580438+0000", 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0056045180000000002, 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069932809999999996, 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28760 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580438+0000", 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580438+0000", 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580437+0000", 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580443+0000", 2024-09-13T09:18:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4400000000000002e-06 2024-09-13T09:18:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580465+0000", 2024-09-13T09:18:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1651e-05 2024-09-13T09:18:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580480+0000", 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5594e-05 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28920 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.585741+0000", 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.00030147099999999999, 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017401949999999999, 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28920 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585741+0000", 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585741+0000", 2024-09-13T09:18:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585740+0000", 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585746+0000", 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9660000000000001e-06 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585763+0000", 2024-09-13T09:18:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7102999999999999e-05 2024-09-13T09:18:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585776+0000", 2024-09-13T09:18:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2918999999999999e-05 2024-09-13T09:18:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28761 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.580504+0000", 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0055382560000000001, 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0070372519999999999, 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28761 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580504+0000", 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580504+0000", 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580503+0000", 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580509+0000", 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3029999999999996e-06 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580526+0000", 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7071e-05 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580540+0000", 2024-09-13T09:18:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3283e-05 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28921 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.585787+0000", 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.00025499099999999999, 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001821076, 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28921 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585787+0000", 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585787+0000", 2024-09-13T09:18:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585786+0000", 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585792+0000", 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585807+0000", 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5668e-05 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585820+0000", 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2490999999999999e-05 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28762 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.580551+0000", 2024-09-13T09:18:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0054915759999999997, 2024-09-13T09:18:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0071272979999999998, 2024-09-13T09:18:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28762 2024-09-13T09:18:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580551+0000", 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580551+0000", 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580550+0000", 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580555+0000", 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6779999999999997e-06 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580571+0000", 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5753999999999999e-05 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580584+0000", 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3020000000000001e-05 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28922 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.585845+0000", 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.000196932, 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001866737, 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28922 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585845+0000", 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585845+0000", 2024-09-13T09:18:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585844+0000", 2024-09-13T09:18:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585850+0000", 2024-09-13T09:18:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7819999999999999e-06 2024-09-13T09:18:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585872+0000", 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1912e-05 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585888+0000", 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6308e-05 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28763 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.580594+0000", 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0054477789999999998, 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0071797559999999998, 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28763 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580594+0000", 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580594+0000", 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580594+0000", 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580622+0000", 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8479999999999998e-05 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580646+0000", 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3509000000000001e-05 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580662+0000", 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6025e-05 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28923 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.585902+0000", 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.00013999499999999999, 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019017820000000001, 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28923 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585902+0000", 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585902+0000", 2024-09-13T09:18:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585901+0000", 2024-09-13T09:18:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585932+0000", 2024-09-13T09:18:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0290999999999998e-05 2024-09-13T09:18:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585980+0000", 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8316000000000003e-05 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.585995+0000", 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5046e-05 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28764 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.580674+0000", 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0053682460000000001, 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0071639670000000003, 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28764 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580674+0000", 2024-09-13T09:18:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580674+0000", 2024-09-13T09:18:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580673+0000", 2024-09-13T09:18:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580678+0000", 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4310000000000002e-06 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580694+0000", 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5654999999999998e-05 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580709+0000", 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4888e-05 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28924 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.586009+0000", 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.3722e-05, 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0018586849999999999, 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28924 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586009+0000", 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586009+0000", 2024-09-13T09:18:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586007+0000", 2024-09-13T09:18:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586014+0000", 2024-09-13T09:18:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2489999999999998e-06 2024-09-13T09:18:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586030+0000", 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6342e-05 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586043+0000", 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.296e-05 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28765 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.580721+0000", 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0053211980000000001, 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0072038299999999996, 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28765 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580721+0000", 2024-09-13T09:18:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580721+0000", 2024-09-13T09:18:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580720+0000", 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580725+0000", 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3850000000000001e-06 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580741+0000", 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6053999999999999e-05 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580760+0000", 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8329e-05 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28925 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.586076+0000", 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9999661, 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001909832, 2024-09-13T09:18:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28925 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586076+0000", 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586076+0000", 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586075+0000", 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586081+0000", 2024-09-13T09:18:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1680000000000001e-06 2024-09-13T09:18:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586099+0000", 2024-09-13T09:18:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7703e-05 2024-09-13T09:18:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586127+0000", 2024-09-13T09:18:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8248000000000001e-05 2024-09-13T09:18:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28766 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.580792+0000", 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0052504270000000002, 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0072618429999999996, 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28766 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580792+0000", 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580792+0000", 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580791+0000", 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580797+0000", 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2230000000000004e-06 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580818+0000", 2024-09-13T09:18:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0837000000000001e-05 2024-09-13T09:18:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580831+0000", 2024-09-13T09:18:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3351e-05 2024-09-13T09:18:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28926 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.586140+0000", 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9999022, 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019643460000000001, 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28926 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586140+0000", 2024-09-13T09:18:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586140+0000", 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586139+0000", 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586145+0000", 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2580000000000001e-06 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586162+0000", 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6265000000000001e-05 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586178+0000", 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6303e-05 2024-09-13T09:18:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28767 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.580851+0000", 2024-09-13T09:18:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0051916619999999997, 2024-09-13T09:18:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0072906519999999999, 2024-09-13T09:18:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28767 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580851+0000", 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580851+0000", 2024-09-13T09:18:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580850+0000", 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580855+0000", 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0170000000000004e-06 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580871+0000", 2024-09-13T09:18:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6056999999999999e-05 2024-09-13T09:18:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580886+0000", 2024-09-13T09:18:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5675000000000002e-05 2024-09-13T09:18:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28927 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.586189+0000", 2024-09-13T09:18:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9998536, 2024-09-13T09:18:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001983046, 2024-09-13T09:18:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28927 2024-09-13T09:18:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586189+0000", 2024-09-13T09:18:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586189+0000", 2024-09-13T09:18:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586188+0000", 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586194+0000", 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6080000000000003e-06 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586209+0000", 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5152e-05 2024-09-13T09:18:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586222+0000", 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2985e-05 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28768 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.580898+0000", 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0051443540000000003, 2024-09-13T09:18:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073035779999999998, 2024-09-13T09:18:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28768 2024-09-13T09:18:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580898+0000", 2024-09-13T09:18:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580898+0000", 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580897+0000", 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580902+0000", 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0939999999999998e-06 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580920+0000", 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7717e-05 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.580933+0000", 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3557000000000001e-05 2024-09-13T09:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:28928 3.3 3.e4b66cc3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.586233+0000", 2024-09-13T09:18:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9998093, 2024-09-13T09:18:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002000206, 2024-09-13T09:18:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 28928 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586233+0000", 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586233+0000", 2024-09-13T09:18:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586232+0000", 2024-09-13T09:18:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586237+0000", 2024-09-13T09:18:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6289999999999998e-06 2024-09-13T09:18:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586257+0000", 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.942e-05 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.586272+0000", 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:16.759 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-13T09:18:16.759 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 47 2024-09-13T09:18:16.759 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:16.761 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:2814 read 0x167d731 thru 0x168afdc (0xd8ac bytes) 2024-09-13T09:18:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:2815 writesame 0x592cb90 thru 0x593a14f data_size 0x90(0xd5c0 bytes) 2024-09-13T09:18:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:2816 write 0x4d7c9ae thru 0x4d7ca74 (0xc7 bytes) 2024-09-13T09:18:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:2818 punch from 0x9b163b9 to 0x9b224f2, (0xc139 bytes) 2024-09-13T09:18:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:2819 read 0x75aa5f1 thru 0x75b813a (0xdb4a bytes) 2024-09-13T09:18:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:2821 writesame 0x1f2104 thru 0x1f4f7c data_size 0x1(0x2e79 bytes) 2024-09-13T09:18:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:2822 punch from 0x7cbec3d to 0x7ccb9fc, (0xcdbf bytes) 2024-09-13T09:18:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:2825 punch from 0x36ee90a to 0x36f7fb7, (0x96ad bytes) 2024-09-13T09:18:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:2826 write 0x8b16f41 thru 0x8b1d2f1 (0x63b1 bytes) 2024-09-13T09:18:16.762 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:16.759+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:2827 write 0x35b6ff thru 0x35e754 (0x3056 bytes) 2024-09-13T09:18:16.828 INFO:teuthology.orchestra.run.smithi032.stdout:2829 punch from 0xbf671af to 0xbf7027e, (0x90cf bytes) 2024-09-13T09:18:16.828 INFO:teuthology.orchestra.run.smithi032.stdout:2830 write 0xa3d97f9 thru 0xa3da7ea (0xff2 bytes) 2024-09-13T09:18:16.830 INFO:teuthology.orchestra.run.smithi032.stdout:2831 read 0x3b83a9 thru 0x3bd561 (0x51b9 bytes) 2024-09-13T09:18:16.831 INFO:teuthology.orchestra.run.smithi032.stdout:2832 trunc from 0xc804fd3 to 0xcd875b4 2024-09-13T09:18:16.842 INFO:teuthology.orchestra.run.smithi032.stdout:2833 trunc from 0xcd875b4 to 0x941e44f 2024-09-13T09:18:16.861 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:16.858+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:16.863 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:16.867 INFO:teuthology.orchestra.run.smithi032.stdout:2837 trunc from 0x941e44f to 0xb9a9386 2024-09-13T09:18:16.873 INFO:teuthology.orchestra.run.smithi032.stdout:2838 writesame 0x35019be thru 0x350acbc data_size 0xb(0x92ff bytes) 2024-09-13T09:18:16.938 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:16.938 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:16.938 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:16.938 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:16.938 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:16.954 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:16.960 INFO:teuthology.orchestra.run.smithi032.stdout:2839 read 0x76102f1 thru 0x7618434 (0x8144 bytes) 2024-09-13T09:18:16.960 INFO:teuthology.orchestra.run.smithi032.stdout:2840 punch from 0xa99f8fb to 0xa9a7396, (0x7a9b bytes) 2024-09-13T09:18:16.961 INFO:teuthology.orchestra.run.smithi032.stdout:2842 read 0x761e3ed thru 0x762425c (0x5e70 bytes) 2024-09-13T09:18:16.961 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:16.959+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:16.962 INFO:teuthology.orchestra.run.smithi032.stdout:2844 read 0xb05732b thru 0xb060ff8 (0x9cce bytes) 2024-09-13T09:18:16.962 INFO:teuthology.orchestra.run.smithi032.stdout:2845 writesame 0x1e64814 thru 0x1e66738 data_size 0x11(0x1f25 bytes) 2024-09-13T09:18:16.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:18:16.981 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:16.981 INFO:teuthology.orchestra.run.smithi032.stdout:2847 write 0x2a7b90e thru 0x2a8b321 (0xfa14 bytes) 2024-09-13T09:18:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:2849 punch from 0x6af8852 to 0x6af9892, (0x1040 bytes) 2024-09-13T09:18:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:2851 write 0x97032fa thru 0x9708135 (0x4e3c bytes) 2024-09-13T09:18:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:2853 write 0x1f2b92b thru 0x1f30cfa (0x53d0 bytes) 2024-09-13T09:18:17.058 INFO:teuthology.orchestra.run.smithi032.stdout:2856 write 0x9510b thru 0xa4a0d (0xf903 bytes) 2024-09-13T09:18:17.062 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:17.058+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:17.079 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:17.079 INFO:teuthology.orchestra.run.smithi032.stdout:2858 read 0x2359deb thru 0x23639cb (0x9be1 bytes) 2024-09-13T09:18:17.080 INFO:teuthology.orchestra.run.smithi032.stdout:2860 read 0x2be9c00 thru 0x2bf1d8e (0x818f bytes) 2024-09-13T09:18:17.081 INFO:teuthology.orchestra.run.smithi032.stdout:2861 read 0x323b7bf thru 0x3242feb (0x782d bytes) 2024-09-13T09:18:17.081 INFO:teuthology.orchestra.run.smithi032.stdout:2862 write 0x7a7772c thru 0x7a79447 (0x1d1c bytes) 2024-09-13T09:18:17.113 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:18:17.122 INFO:teuthology.orchestra.run.smithi032.stdout:2864 read 0xb7f2ce4 thru 0xb801d42 (0xf05f bytes) 2024-09-13T09:18:17.122 INFO:teuthology.orchestra.run.smithi032.stdout:2865 punch from 0xfb3ce4 to 0xfbd8f5, (0x9c11 bytes) 2024-09-13T09:18:17.122 INFO:teuthology.orchestra.run.smithi032.stdout:2866 writesame 0x5ef2dad thru 0x5f00a5d data_size 0x7(0xdcb1 bytes) 2024-09-13T09:18:17.156 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:17.156 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4388 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168857267999996, 2024-09-13T09:18:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069349509000000004, 2024-09-13T09:18:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4388 2024-09-13T09:18:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985769+0000", 2024-09-13T09:18:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985775+0000", 2024-09-13T09:18:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5230000000000001e-06 2024-09-13T09:18:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985794+0000", 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8425999999999999e-05 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985809+0000", 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.486e-05 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.054965+0000", 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069156463000000001 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055036+0000", 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0780999999999999e-05 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055120+0000", 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3486000000000007e-05 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4389 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168806744999998, 2024-09-13T09:18:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069473833999999998, 2024-09-13T09:18:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4389 2024-09-13T09:18:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985820+0000", 2024-09-13T09:18:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985859+0000", 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9279999999999997e-05 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985881+0000", 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2116999999999998e-05 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985896+0000", 2024-09-13T09:18:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4698999999999999e-05 2024-09-13T09:18:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055152+0000", 2024-09-13T09:18:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069256568000000004 2024-09-13T09:18:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055220+0000", 2024-09-13T09:18:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8255000000000003e-05 2024-09-13T09:18:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055294+0000", 2024-09-13T09:18:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3858999999999999e-05 2024-09-13T09:18:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4390 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168711133000002, 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069563635999999998, 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4390 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985915+0000", 2024-09-13T09:18:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985922+0000", 2024-09-13T09:18:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:18:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985941+0000", 2024-09-13T09:18:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8260000000000001e-05 2024-09-13T09:18:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985955+0000", 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4409999999999999e-05 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055325+0000", 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069370290000000001 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055395+0000", 2024-09-13T09:18:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0158000000000002e-05 2024-09-13T09:18:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055480+0000", 2024-09-13T09:18:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4340999999999998e-05 2024-09-13T09:18:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168660066000001, 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:18:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:18:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:18:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168611329999997, 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:18:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:18:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:18:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:18:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:18:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:18:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168543661000001, 2024-09-13T09:18:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:18:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:18:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:18:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:18:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:18:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:18:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:18:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:18:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:18:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:18:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:18:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:18:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:18:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:18:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:18:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168403452, 2024-09-13T09:18:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:18:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:18:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:18:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:18:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:18:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:18:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:18:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:18:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:18:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168312585999999, 2024-09-13T09:18:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:18:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:18:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:18:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168237726000001, 2024-09-13T09:18:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:18:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:18:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:18:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:18:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:18:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168166550000002, 2024-09-13T09:18:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:18:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:18:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:18:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:18:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:18:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:18:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:18:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:18:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:18:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:18:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168117496999997, 2024-09-13T09:18:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:18:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:18:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:18:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:18:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:18:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:18:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:18:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:18:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:18:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168055760999998, 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:18:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.168006091000002, 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:18:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:18:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:18:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.167957571999999, 2024-09-13T09:18:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:18:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:18:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.167909176000002, 2024-09-13T09:18:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:18:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:18:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:18:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:18:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:18:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.167861889000001, 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:18:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:18:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:18:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:18:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:18:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:18:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:18:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:18:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:18:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:18:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.167778487, 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:18:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:18:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:18:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:18:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:18:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:18:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:18:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.167704761000003, 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:18:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:18:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:18:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:18:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.167636637999998, 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:18:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:18:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:18:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:18:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:18:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.167586171000004, 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:18:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:18:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:18:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:18:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:18:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:18:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:18:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:17.263 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:17.264 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:17.265 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:17.265 INFO:teuthology.orchestra.run.smithi032.stdout:2867 write 0x2e1e63d thru 0x2e220ea (0x3aae bytes) 2024-09-13T09:18:17.266 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:17.266 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:17.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:17.267 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:17.258+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:2868 write 0xd6a8c04 thru 0xd6a9664 (0xa61 bytes) 2024-09-13T09:18:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:2872 read 0x4d50692 thru 0x4d510bf (0xa2e bytes) 2024-09-13T09:18:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:2874 write 0x5f94bc5 thru 0x5fa488c (0xfcc8 bytes) 2024-09-13T09:18:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:2875 trunc from 0xd6a9665 to 0xd2efa04 2024-09-13T09:18:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:2876 punch from 0x81306ba to 0x813afc3, (0xa909 bytes) 2024-09-13T09:18:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:2877 write 0x850a65a thru 0x851191f (0x72c6 bytes) 2024-09-13T09:18:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:2878 write 0x42b8f97 thru 0x42c774b (0xe7b5 bytes) 2024-09-13T09:18:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:17.269 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:17.285 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:17.285 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:17.308 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:17.308 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:17.342 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:17.344 INFO:teuthology.orchestra.run.smithi032.stdout:2879 read 0x5f38fde thru 0x5f39a5f (0xa82 bytes) 2024-09-13T09:18:17.361 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:17.361 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:17.362 INFO:teuthology.orchestra.run.smithi032.stdout:2880 trunc from 0xd2efa04 to 0x850abd 2024-09-13T09:18:17.365 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:17.362+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:17.394 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:17.394 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:17.450 INFO:teuthology.orchestra.run.smithi032.stdout:2881 writesame 0x56aac35 thru 0x56b90a7 data_size 0x1(0xe473 bytes) 2024-09-13T09:18:17.451 INFO:teuthology.orchestra.run.smithi032.stdout:2882 write 0x50c994b thru 0x50cf54e (0x5c04 bytes) 2024-09-13T09:18:17.452 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:17.452 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:17.460 INFO:teuthology.orchestra.run.smithi032.stdout:2886 trunc from 0x56b90a8 to 0x5310f5f 2024-09-13T09:18:17.466 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:17.462+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:17.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:17.483 INFO:teuthology.orchestra.run.smithi032.stdout:2887 writesame 0xa14e6e8 thru 0xa1579b4 data_size 0x1(0x92cd bytes) 2024-09-13T09:18:17.483 INFO:teuthology.orchestra.run.smithi032.stdout:2888 write 0x3eb4315 thru 0x3eba305 (0x5ff1 bytes) 2024-09-13T09:18:17.501 INFO:teuthology.orchestra.run.smithi032.stdout:2889 write 0xbeb405b thru 0xbebad80 (0x6d26 bytes) 2024-09-13T09:18:17.510 INFO:teuthology.orchestra.run.smithi032.stdout:2890 read 0x91e8912 thru 0x91e9f53 (0x1642 bytes) 2024-09-13T09:18:17.510 INFO:teuthology.orchestra.run.smithi032.stdout:2891 write 0x1278841 thru 0x12882cb (0xfa8b bytes) 2024-09-13T09:18:17.519 INFO:teuthology.orchestra.run.smithi032.stdout:2892 writesame 0x5db47c thru 0x5e250a data_size 0x55(0x708f bytes) 2024-09-13T09:18:17.533 INFO:teuthology.orchestra.run.smithi032.stdout:2893 trunc from 0xbebad81 to 0x37ff7a3 2024-09-13T09:18:17.550 INFO:teuthology.orchestra.run.smithi032.stdout:2894 read 0x15ecefb thru 0x15edaaf (0xbb5 bytes) 2024-09-13T09:18:17.550 INFO:teuthology.orchestra.run.smithi032.stdout:2895 read 0x335bb07 thru 0x336802f (0xc529 bytes) 2024-09-13T09:18:17.550 INFO:teuthology.orchestra.run.smithi032.stdout:2896 write 0x244f379 thru 0x2457720 (0x83a8 bytes) 2024-09-13T09:18:17.559 INFO:teuthology.orchestra.run.smithi032.stdout:2897 read 0x64f580 thru 0x65e87d (0xf2fe bytes) 2024-09-13T09:18:17.567 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:17.567+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:17.568 INFO:teuthology.orchestra.run.smithi032.stdout:2899 writesame 0x4a1c098 thru 0x4a2152a data_size 0x7(0x5493 bytes) 2024-09-13T09:18:17.568 INFO:teuthology.orchestra.run.smithi032.stdout:2900 read 0x3af5b9d thru 0x3b03f4c (0xe3b0 bytes) 2024-09-13T09:18:17.568 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:17.568 INFO:teuthology.orchestra.run.smithi032.stdout:2902 punch from 0x2a6b2e5 to 0x2a70fe0, (0x5cfb bytes) 2024-09-13T09:18:17.580 INFO:teuthology.orchestra.run.smithi032.stdout:2903 write 0x8e0935e thru 0x8e0e039 (0x4cdc bytes) 2024-09-13T09:18:17.588 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:17.588 INFO:teuthology.orchestra.run.smithi032.stdout:2905 writesame 0x511703b thru 0x51232c1 data_size 0x1(0xc287 bytes) 2024-09-13T09:18:17.611 INFO:teuthology.orchestra.run.smithi032.stdout:2906 trunc from 0x8e0e03a to 0x41c6449 2024-09-13T09:18:17.642 INFO:teuthology.orchestra.run.smithi032.stdout:2908 write 0xa19c049 thru 0xa1a0657 (0x460f bytes) 2024-09-13T09:18:17.651 INFO:teuthology.orchestra.run.smithi032.stdout:2909 read 0x1d15285 thru 0x1d25041 (0xfdbd bytes) 2024-09-13T09:18:17.652 INFO:teuthology.orchestra.run.smithi032.stdout:2910 trunc from 0xa1a0658 to 0xa43e81e 2024-09-13T09:18:17.653 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:17.653 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:17.664 INFO:teuthology.orchestra.run.smithi032.stdout:2911 read 0xa19728f thru 0xa1a2336 (0xb0a8 bytes) 2024-09-13T09:18:17.665 INFO:teuthology.orchestra.run.smithi032.stdout:2913 read 0x5a415f thru 0x5b1023 (0xcec5 bytes) 2024-09-13T09:18:17.667 INFO:teuthology.orchestra.run.smithi032.stdout:2914 write 0x34860a4 thru 0x34912fc (0xb259 bytes) 2024-09-13T09:18:17.667 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:17.667+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:17.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:17.685 INFO:teuthology.orchestra.run.smithi032.stdout:2915 trunc from 0xa43e81e to 0x88e61e9 2024-09-13T09:18:17.700 INFO:teuthology.orchestra.run.smithi032.stdout:2917 read 0x5a35b9b thru 0x5a3aa92 (0x4ef8 bytes) 2024-09-13T09:18:17.700 INFO:teuthology.orchestra.run.smithi032.stdout:2918 punch from 0x4d344bb to 0x4d3a3ad, (0x5ef2 bytes) 2024-09-13T09:18:17.700 INFO:teuthology.orchestra.run.smithi032.stdout:2919 read 0x1dabde1 thru 0x1db87fb (0xca1b bytes) 2024-09-13T09:18:17.700 INFO:teuthology.orchestra.run.smithi032.stdout:2920 read 0x4e68d62 thru 0x4e73a64 (0xad03 bytes) 2024-09-13T09:18:17.700 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:17.701 INFO:teuthology.orchestra.run.smithi032.stdout:2922 writesame 0xd75a8f thru 0xd7c515 data_size 0x1(0x6a87 bytes) 2024-09-13T09:18:17.720 INFO:teuthology.orchestra.run.smithi032.stdout:2923 write 0xaf88ff3 thru 0xaf8d91c (0x492a bytes) 2024-09-13T09:18:17.743 INFO:teuthology.orchestra.run.smithi032.stdout:2925 trunc from 0xaf8d91d to 0x40eb814 2024-09-13T09:18:17.768 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:17.767+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:17.772 INFO:teuthology.orchestra.run.smithi032.stdout:2926 trunc from 0x40eb814 to 0xe0a46ac 2024-09-13T09:18:17.784 INFO:teuthology.orchestra.run.smithi032.stdout:2927 read 0x7cc3980 thru 0x7cd1737 (0xddb8 bytes) 2024-09-13T09:18:17.794 INFO:teuthology.orchestra.run.smithi032.stdout:2928 trunc from 0xe0a46ac to 0x7af3b26 2024-09-13T09:18:17.799 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:17.799 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:17.808 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:17.808 INFO:teuthology.orchestra.run.smithi032.stdout:2932 write 0x2e4f140 thru 0x2e50cc5 (0x1b86 bytes) 2024-09-13T09:18:17.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:17.820 INFO:teuthology.orchestra.run.smithi032.stdout:2933 trunc from 0x7af3b26 to 0x532e461 2024-09-13T09:18:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:2935 punch from 0x279ca51 to 0x27ab2c4, (0xe873 bytes) 2024-09-13T09:18:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:2936 read 0xbc1f8d thru 0xbc2bc3 (0xc37 bytes) 2024-09-13T09:18:17.835 INFO:teuthology.orchestra.run.smithi032.stdout:2938 read 0x18f41d7 thru 0x1901a67 (0xd891 bytes) 2024-09-13T09:18:17.859 INFO:teuthology.orchestra.run.smithi032.stdout:2939 write 0xa94617c thru 0xa94b588 (0x540d bytes) 2024-09-13T09:18:17.869 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:17.867+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:17.877 INFO:teuthology.orchestra.run.smithi032.stdout:2940 read 0xa5d9ca6 thru 0xa5da99d (0xcf8 bytes) 2024-09-13T09:18:17.894 INFO:teuthology.orchestra.run.smithi032.stdout:2941 write 0xd77e626 thru 0xd7897fb (0xb1d6 bytes) 2024-09-13T09:18:17.916 INFO:teuthology.orchestra.run.smithi032.stdout:2942 read 0xc3104fe thru 0xc31598b (0x548e bytes) 2024-09-13T09:18:17.916 INFO:teuthology.orchestra.run.smithi032.stdout:2945 write 0x9517462 thru 0x9524d3e (0xd8dd bytes) 2024-09-13T09:18:17.937 INFO:teuthology.orchestra.run.smithi032.stdout:2947 punch from 0x5127838 to 0x51305c8, (0x8d90 bytes) 2024-09-13T09:18:17.938 INFO:teuthology.orchestra.run.smithi032.stdout:2948 writesame 0x4099f0e thru 0x40a5d86 data_size 0x1(0xbe79 bytes) 2024-09-13T09:18:17.945 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:17.945 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:17.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:17.981 INFO:teuthology.orchestra.run.smithi032.stdout:2949 write 0xdeb6439 thru 0xdec53af (0xef77 bytes) 2024-09-13T09:18:18.002 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:18.002 INFO:teuthology.orchestra.run.smithi032.stdout:2953 write 0xc474407 thru 0xc482062 (0xdc5c bytes) 2024-09-13T09:18:18.009 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:18.006+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:18.021 INFO:teuthology.orchestra.run.smithi032.stdout:2954 write 0xd89edca thru 0xd8addeb (0xf022 bytes) 2024-09-13T09:18:18.025 INFO:teuthology.orchestra.run.smithi032.stdout:2955 read 0x1f67fe9 thru 0x1f733ea (0xb402 bytes) 2024-09-13T09:18:18.026 INFO:teuthology.orchestra.run.smithi032.stdout:2956 read 0xdc5558f thru 0xdc5b1a9 (0x5c1b bytes) 2024-09-13T09:18:18.026 INFO:teuthology.orchestra.run.smithi032.stdout:2957 write 0x20c74da thru 0x20ce6f1 (0x7218 bytes) 2024-09-13T09:18:18.043 INFO:teuthology.orchestra.run.smithi032.stdout:2958 writesame 0x16808d8 thru 0x1684a24 data_size 0x1(0x414d bytes) 2024-09-13T09:18:18.055 INFO:teuthology.orchestra.run.smithi032.stdout:2959 read 0xbae49ba thru 0xbaf2785 (0xddcc bytes) 2024-09-13T09:18:18.070 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:18.071+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-13T09:18:18.082 INFO:teuthology.orchestra.run.smithi032.stdout:2960 trunc from 0xdec53b0 to 0xa18b15 2024-09-13T09:18:18.105 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:18.105 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:18.127 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:18.148 INFO:teuthology.orchestra.run.smithi032.stdout:2962 trunc from 0xa18b15 to 0x8926f42 2024-09-13T09:18:18.168 INFO:teuthology.orchestra.run.smithi032.stdout:2963 writesame 0x9a98a08 thru 0x9aa3b13 data_size 0x12(0xb10c bytes) 2024-09-13T09:18:18.168 INFO:teuthology.orchestra.run.smithi032.stdout:2964 write 0x68ed1c8 thru 0x68f3c21 (0x6a5a bytes) 2024-09-13T09:18:18.174 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:18.170+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:18.181 INFO:teuthology.orchestra.run.smithi032.stdout:2965 read 0x896a187 thru 0x8979c28 (0xfaa2 bytes) 2024-09-13T09:18:18.189 INFO:teuthology.orchestra.run.smithi032.stdout:2967 trunc from 0x9aa3b14 to 0xe5ae5d4 2024-09-13T09:18:18.199 INFO:teuthology.orchestra.run.smithi032.stdout:2968 punch from 0xd7c10ae to 0xd7d076c, (0xf6be bytes) 2024-09-13T09:18:18.200 INFO:teuthology.orchestra.run.smithi032.stdout:2969 writesame 0x9bf6f2e thru 0x9bfcf86 data_size 0x1(0x6059 bytes) 2024-09-13T09:18:18.220 INFO:teuthology.orchestra.run.smithi032.stdout:2970 writesame 0x7c42ba2 thru 0x7c4de19 data_size 0x4(0xb278 bytes) 2024-09-13T09:18:18.246 INFO:teuthology.orchestra.run.smithi032.stdout:2971 trunc from 0xe5ae5d4 to 0xb2bdb81 2024-09-13T09:18:18.258 INFO:teuthology.orchestra.run.smithi032.stdout:2972 read 0x8f13523 thru 0x8f186da (0x51b8 bytes) 2024-09-13T09:18:18.267 INFO:teuthology.orchestra.run.smithi032.stdout:2974 trunc from 0xb2bdb81 to 0x692b0c0 2024-09-13T09:18:18.273 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:18.271+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:18.291 INFO:teuthology.orchestra.run.smithi032.stdout:2975 punch from 0x2ca50d0 to 0x2ca94be, (0x43ee bytes) 2024-09-13T09:18:18.301 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:18.302 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:18.304 INFO:teuthology.orchestra.run.smithi032.stdout:2976 writesame 0x6a053cc thru 0x6a09ff7 data_size 0x41(0x4c2c bytes) 2024-09-13T09:18:18.317 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:18.323 INFO:teuthology.orchestra.run.smithi032.stdout:2977 write 0x9769234 thru 0x976d681 (0x444e bytes) 2024-09-13T09:18:18.335 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:18.335 INFO:teuthology.orchestra.run.smithi032.stdout:2980 read 0x49b4c0d thru 0x49b9e99 (0x528d bytes) 2024-09-13T09:18:18.335 INFO:teuthology.orchestra.run.smithi032.stdout:2981 punch from 0x39a3c44 to 0x39a41b6, (0x572 bytes) 2024-09-13T09:18:18.335 INFO:teuthology.orchestra.run.smithi032.stdout:2982 read 0x7e48a5f thru 0x7e524f4 (0x9a96 bytes) 2024-09-13T09:18:18.335 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:18.335 INFO:teuthology.orchestra.run.smithi032.stdout:2984 write 0x376007e thru 0x3766827 (0x67aa bytes) 2024-09-13T09:18:18.353 INFO:teuthology.orchestra.run.smithi032.stdout:2987 write 0x44868ee thru 0x448a603 (0x3d16 bytes) 2024-09-13T09:18:18.365 INFO:teuthology.orchestra.run.smithi032.stdout:2989 write 0x6de53e5 thru 0x6def73c (0xa358 bytes) 2024-09-13T09:18:18.373 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:18.370+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:18.397 INFO:teuthology.orchestra.run.smithi032.stdout:2990 write 0xb1b4b10 thru 0xb1bcda5 (0x8296 bytes) 2024-09-13T09:18:18.416 INFO:teuthology.orchestra.run.smithi032.stdout:2991 flatten 2024-09-13T09:18:18.428 INFO:teuthology.orchestra.run.smithi032.stdout:2992 read 0xa47826f thru 0xa47b6c8 (0x345a bytes) 2024-09-13T09:18:18.453 INFO:teuthology.orchestra.run.smithi032.stdout:2993 write 0xdacbdd3 thru 0xdad6326 (0xa554 bytes) 2024-09-13T09:18:18.456 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:18.457 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:18.472 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:18.473 INFO:teuthology.orchestra.run.smithi032.stdout:2994 read 0x57762be thru 0x5777ecd (0x1c10 bytes) 2024-09-13T09:18:18.474 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:18.471+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:18.500 INFO:teuthology.orchestra.run.smithi032.stdout:2998 trunc from 0xdad6327 to 0x1e8b765 2024-09-13T09:18:18.540 INFO:teuthology.orchestra.run.smithi032.stdout:2999 clone 20 order 20 su 16384 sc 15 2024-09-13T09:18:18.575 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:18.575+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:18.644 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:18.644 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:18.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:18.676 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:18.675+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:18.776 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:18.776 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:18.777 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:18.770+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:18.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:18.878 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:18.874+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:18.926 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:18.926 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:18.940 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:18.979 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:18.974+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:19.067 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:19.067 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:19.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:19.081 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:19.078+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:19.181 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:19.179+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:19.213 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:19.213 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:19.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:19.282 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:19.278+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:19.358 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-13T09:18:19.383 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:19.382+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:19.418 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:19.418 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:19.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:19.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:19.509 INFO:teuthology.orchestra.run.smithi032.stdout:3000 write 0x3fab448 thru 0x3fabcb1 (0x86a bytes) 2024-09-13T09:18:19.510 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:19.507+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:19.515 INFO:teuthology.orchestra.run.smithi032.stdout:3002 write 0x34ec9d4 thru 0x34f462a (0x7c57 bytes) 2024-09-13T09:18:19.524 INFO:teuthology.orchestra.run.smithi032.stdout:3003 punch from 0x45ec51 to 0x46b4b3, (0xc862 bytes) 2024-09-13T09:18:19.524 INFO:teuthology.orchestra.run.smithi032.stdout:3004 read 0x2a2cf74 thru 0x2a3ac57 (0xdce4 bytes) 2024-09-13T09:18:19.539 INFO:teuthology.orchestra.run.smithi032.stdout:3006 writesame 0x4d3de9a thru 0x4d4b408 data_size 0x27(0xd56f bytes) 2024-09-13T09:18:19.539 INFO:teuthology.orchestra.run.smithi032.stdout:3008 punch from 0x2537ecf to 0x25470d8, (0xf209 bytes) 2024-09-13T09:18:19.541 INFO:teuthology.orchestra.run.smithi032.stdout:3013 trunc from 0x4d4b409 to 0x36b0fb6 2024-09-13T09:18:19.563 INFO:teuthology.orchestra.run.smithi032.stdout:3014 punch from 0x13ba72e to 0x13c8a74, (0xe346 bytes) 2024-09-13T09:18:19.563 INFO:teuthology.orchestra.run.smithi032.stdout:3015 read 0x2519781 thru 0x251ad34 (0x15b4 bytes) 2024-09-13T09:18:19.563 INFO:teuthology.orchestra.run.smithi032.stdout:3016 trunc from 0x36b0fb6 to 0x36db10c 2024-09-13T09:18:19.574 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:19.574 INFO:teuthology.orchestra.run.smithi032.stdout:3018 punch from 0x16cc7b2 to 0x16cc91c, (0x16a bytes) 2024-09-13T09:18:19.581 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:19.582 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:19.585 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:19.582+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:19.595 INFO:teuthology.orchestra.run.smithi032.stdout:3020 write 0xa17d10b thru 0xa17d474 (0x36a bytes) 2024-09-13T09:18:19.599 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:19.600 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:19.601 INFO:teuthology.orchestra.run.smithi032.stdout:3021 read 0x6a688fc thru 0x6a6fc58 (0x735d bytes) 2024-09-13T09:18:19.601 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:19.601 INFO:teuthology.orchestra.run.smithi032.stdout:3023 read 0x141f52c thru 0x141ff4e (0xa23 bytes) 2024-09-13T09:18:19.601 INFO:teuthology.orchestra.run.smithi032.stdout:3024 writesame 0x70344ad thru 0x703c1ef data_size 0x15(0x7d43 bytes) 2024-09-13T09:18:19.608 INFO:teuthology.orchestra.run.smithi032.stdout:3025 write 0x47c52bc thru 0x47ca91d (0x5662 bytes) 2024-09-13T09:18:19.614 INFO:teuthology.orchestra.run.smithi032.stdout:3026 read 0x9fb9a2c thru 0x9fba2a7 (0x87c bytes) 2024-09-13T09:18:19.615 INFO:teuthology.orchestra.run.smithi032.stdout:3027 trunc from 0xa17d475 to 0x8f7ea5a 2024-09-13T09:18:19.618 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:19.619 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:19.619 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:19.619 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:19.619 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:19.619 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:19.635 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:19.636 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:19.636 INFO:teuthology.orchestra.run.smithi032.stdout:3028 trunc from 0x8f7ea5a to 0x9172799 2024-09-13T09:18:19.657 INFO:teuthology.orchestra.run.smithi032.stdout:3029 trunc from 0x9172799 to 0x266b5d5 2024-09-13T09:18:19.657 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:19.686 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:19.682+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:19.687 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:19.688 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:19.701 INFO:teuthology.orchestra.run.smithi032.stdout:3030 read 0x13cd40d thru 0x13d0af2 (0x36e6 bytes) 2024-09-13T09:18:19.709 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:19.724 INFO:teuthology.orchestra.run.smithi032.stdout:3031 writesame 0x8fd79b5 thru 0x8fda1bd data_size 0x1(0x2809 bytes) 2024-09-13T09:18:19.724 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:19.724 INFO:teuthology.orchestra.run.smithi032.stdout:3034 writesame 0x28c2254 thru 0x28d14f0 data_size 0x67(0xf29d bytes) 2024-09-13T09:18:19.727 INFO:teuthology.orchestra.run.smithi032.stdout:3036 writesame 0x178bf00 thru 0x178ee5e data_size 0x1(0x2f5f bytes) 2024-09-13T09:18:19.733 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:19.739 INFO:teuthology.orchestra.run.smithi032.stdout:3037 writesame 0x3116db9 thru 0x3117f7f data_size 0x3(0x11c7 bytes) 2024-09-13T09:18:19.746 INFO:teuthology.orchestra.run.smithi032.stdout:3039 write 0x29801b9 thru 0x2985276 (0x50be bytes) 2024-09-13T09:18:19.749 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:19.749 INFO:teuthology.orchestra.run.smithi032.stdout:3041 read 0x1dcbf03 thru 0x1dd657c (0xa67a bytes) 2024-09-13T09:18:19.749 INFO:teuthology.orchestra.run.smithi032.stdout:3042 read 0x5bac480 thru 0x5bb87cc (0xc34d bytes) 2024-09-13T09:18:19.750 INFO:teuthology.orchestra.run.smithi032.stdout:3043 read 0x34e39db thru 0x34f2450 (0xea76 bytes) 2024-09-13T09:18:19.750 INFO:teuthology.orchestra.run.smithi032.stdout:3044 write 0x4dc1f2c thru 0x4dccc4e (0xad23 bytes) 2024-09-13T09:18:19.768 INFO:teuthology.orchestra.run.smithi032.stdout:3045 trunc from 0x8fda1be to 0x3c1efa4 2024-09-13T09:18:19.771 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:19.771 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:19.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:19.771 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:19.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.771 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:18:19.771 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:18:19.771 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.696658531000001, 2024-09-13T09:18:19.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:18:19.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.771 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:19.771 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:18:19.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:18:19.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29220 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1052672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.593818+0000", 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1768447769999999, 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032623085000000003, 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29220 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593818+0000", 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593818+0000", 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593817+0000", 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593823+0000", 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8370000000000002e-06 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593840+0000", 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6875999999999999e-05 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593854+0000", 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4014e-05 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626245+0000", 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032391101999999998 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626343+0000", 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7442000000000007e-05 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626441+0000", 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8863000000000005e-05 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29221 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.593865+0000", 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.176797729, 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032785591000000003, 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29221 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593865+0000", 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593865+0000", 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593864+0000", 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:19.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593870+0000", 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.694e-06 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593885+0000", 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5065000000000001e-05 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593897+0000", 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1779e-05 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626478+0000", 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032581242000000003 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626578+0000", 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9919000000000003e-05 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626651+0000", 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3182999999999994e-05 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29222 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1110016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.593907+0000", 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1767557260000001, 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032975155999999999, 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29222 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593907+0000", 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593907+0000", 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593906+0000", 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593912+0000", 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5910000000000003e-06 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593930+0000", 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7918999999999999e-05 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593942+0000", 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1875999999999999e-05 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626684+0000", 2024-09-13T09:18:19.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032742344 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626781+0000", 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6879000000000005e-05 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626883+0000", 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101761 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29223 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1138688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.593953+0000", 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1767105139999998, 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033225364, 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29223 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.792 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593953+0000", 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593953+0000", 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593952+0000", 2024-09-13T09:18:19.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593957+0000", 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593975+0000", 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7671000000000001e-05 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593987+0000", 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2225000000000001e-05 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626925+0000", 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032937578000000002 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627068+0000", 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014354600000000001 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627178+0000", 2024-09-13T09:18:19.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109622 2024-09-13T09:18:19.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.796 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.796 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29224 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1167360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.796 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594001+0000", 2024-09-13T09:18:19.796 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.176661932, 2024-09-13T09:18:19.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033434541999999998, 2024-09-13T09:18:19.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29224 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594001+0000", 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594001+0000", 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594000+0000", 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594023+0000", 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3030000000000001e-05 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594040+0000", 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7108e-05 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594053+0000", 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3185e-05 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627217+0000", 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033163555999999997 2024-09-13T09:18:19.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627329+0000", 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112562 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627436+0000", 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106538 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29225 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1196032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594112+0000", 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.176551313, 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033646220999999997, 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29225 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594112+0000", 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594112+0000", 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594111+0000", 2024-09-13T09:18:19.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594118+0000", 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1829999999999997e-06 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594136+0000", 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8444000000000002e-05 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594164+0000", 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8238999999999998e-05 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627501+0000", 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033336688000000003 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627631+0000", 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000130128 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627758+0000", 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012672200000000001 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29226 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1224704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594176+0000", 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.176486905, 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033850475999999997, 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.805 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29226 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594176+0000", 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594176+0000", 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594175+0000", 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594181+0000", 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2389999999999999e-06 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594197+0000", 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5464999999999998e-05 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594210+0000", 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3071000000000001e-05 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627828+0000", 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033618289000000003 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627938+0000", 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110136 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628027+0000", 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8418999999999995e-05 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29227 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1253376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594221+0000", 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1764424509999998, 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034003004000000003, 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29227 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594221+0000", 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594221+0000", 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594220+0000", 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594225+0000", 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5849999999999999e-06 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594241+0000", 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5542000000000001e-05 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594255+0000", 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4396000000000001e-05 2024-09-13T09:18:19.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628061+0000", 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033806151999999999 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628160+0000", 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8400999999999995e-05 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628224+0000", 2024-09-13T09:18:19.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3938999999999999e-05 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29228 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594266+0000", 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1763968309999999, 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034238179000000001, 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29228 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594266+0000", 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594266+0000", 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594265+0000", 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594286+0000", 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1208000000000001e-05 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594308+0000", 2024-09-13T09:18:19.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1563999999999999e-05 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594322+0000", 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3742e-05 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628275+0000", 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033953637000000002 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628393+0000", 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117543 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628504+0000", 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111474 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29229 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594333+0000", 2024-09-13T09:18:19.818 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1763298349999998, 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034422919000000003, 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29229 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594333+0000", 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594333+0000", 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594332+0000", 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594338+0000", 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594354+0000", 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5727999999999999e-05 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594367+0000", 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2473000000000001e-05 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628558+0000", 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034191322000000003 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628661+0000", 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103405 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628756+0000", 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4943000000000002e-05 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.822 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29230 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1339392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594446+0000", 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1762166110000001, 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034555941, 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29230 2024-09-13T09:18:19.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594446+0000", 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594446+0000", 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594445+0000", 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594454+0000", 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3480000000000004e-06 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594474+0000", 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0438999999999999e-05 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594489+0000", 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4961e-05 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628813+0000", 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034323499 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628920+0000", 2024-09-13T09:18:19.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107293 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629002+0000", 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2575000000000001e-05 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29231 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594501+0000", 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1761616369999999, 2024-09-13T09:18:19.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034737715000000002, 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29231 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594501+0000", 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594501+0000", 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594500+0000", 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594520+0000", 2024-09-13T09:18:19.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9542e-05 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594537+0000", 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7399000000000001e-05 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594551+0000", 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3653e-05 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629069+0000", 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034518189999999997 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629169+0000", 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100171 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629239+0000", 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0005999999999993e-05 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29232 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1396736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594563+0000", 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1761005039999999, 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986323, 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29232 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.832 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594563+0000", 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594563+0000", 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594561+0000", 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594567+0000", 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594583+0000", 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6047000000000001e-05 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594596+0000", 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2619000000000001e-05 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629292+0000", 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034696236999999998 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629420+0000", 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012762599999999999 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629549+0000", 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012909099999999999 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29233 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594607+0000", 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1760563180000001, 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035129889999999997, 2024-09-13T09:18:19.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29233 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594607+0000", 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594607+0000", 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594606+0000", 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594611+0000", 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594627+0000", 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5619000000000001e-05 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594639+0000", 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2398999999999999e-05 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629605+0000", 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034965969 2024-09-13T09:18:19.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629671+0000", 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6099000000000005e-05 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629737+0000", 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5499000000000004e-05 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29234 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594650+0000", 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1760133609999999, 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035235401999999999, 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29234 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594650+0000", 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594650+0000", 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594649+0000", 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594654+0000", 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6280000000000002e-06 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594670+0000", 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6161999999999999e-05 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594697+0000", 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6284000000000001e-05 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629769+0000", 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071946999999999 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629833+0000", 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4721999999999995e-05 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629885+0000", 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1749999999999997e-05 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29235 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1482752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594709+0000", 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1759537760000001, 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035355323000000001, 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.845 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29235 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594709+0000", 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594709+0000", 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594708+0000", 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594715+0000", 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3049999999999998e-06 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594732+0000", 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7229999999999999e-05 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594745+0000", 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2694000000000001e-05 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629923+0000", 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035178216999999998 2024-09-13T09:18:19.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629997+0000", 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3971e-05 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630065+0000", 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7940000000000003e-05 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29236 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1511424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594756+0000", 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1759073020000002, 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035707192999999998, 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29236 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.850 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594756+0000", 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594756+0000", 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594755+0000", 2024-09-13T09:18:19.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594760+0000", 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6960000000000002e-06 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594777+0000", 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6239000000000002e-05 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594789+0000", 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2734e-05 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630196+0000", 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035406919000000002 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630337+0000", 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000140801 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630463+0000", 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012583200000000001 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29237 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594815+0000", 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1758485310000002, 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035904437999999997, 2024-09-13T09:18:19.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29237 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594815+0000", 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594815+0000", 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594813+0000", 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594820+0000", 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1480000000000003e-06 2024-09-13T09:18:19.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594836+0000", 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6379e-05 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594848+0000", 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2231e-05 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630525+0000", 2024-09-13T09:18:19.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035676280999999997 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630631+0000", 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106099 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630719+0000", 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8362999999999994e-05 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29238 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1568768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594914+0000", 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.1757486689999999, 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036012464000000001, 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29238 2024-09-13T09:18:19.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594914+0000", 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594914+0000", 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594913+0000", 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594920+0000", 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4429999999999997e-06 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594940+0000", 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9273e-05 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594974+0000", 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4403000000000002e-05 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630755+0000", 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035780660999999998 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630854+0000", 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9498000000000002e-05 2024-09-13T09:18:19.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:19.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:19.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:19.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630927+0000", 2024-09-13T09:18:19.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2591999999999999e-05 2024-09-13T09:18:19.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.863 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:19.863 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:19.863 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:19.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:19.865 INFO:teuthology.orchestra.run.smithi032.stdout:3046 write 0x8fbbadf thru 0x8fc884e (0xcd70 bytes) 2024-09-13T09:18:19.865 INFO:teuthology.orchestra.run.smithi032.stdout:3047 writesame 0xc8fc16 thru 0xc97439 data_size 0x2c(0x7824 bytes) 2024-09-13T09:18:19.866 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:19.858+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:19.891 INFO:teuthology.orchestra.run.smithi032.stdout:3048 punch from 0x2883ca0 to 0x288f036, (0xb396 bytes) 2024-09-13T09:18:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:3049 trunc from 0x8fc884f to 0x26c4ca8 2024-09-13T09:18:19.965 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:19.963+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:3052 writesame 0xe84718d thru 0xe84fcc5 data_size 0x1(0x8b39 bytes) 2024-09-13T09:18:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:3053 read 0x21fcef3 thru 0x220397d (0x6a8b bytes) 2024-09-13T09:18:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:3054 writesame 0x6c1eda thru 0x6ce478 data_size 0x1(0xc59f bytes) 2024-09-13T09:18:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:20.038 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:20.038 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:20.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:20.064 INFO:teuthology.orchestra.run.smithi032.stdout:3055 write 0x46fabfc thru 0x47050c4 (0xa4c9 bytes) 2024-09-13T09:18:20.066 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:20.067+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:3056 punch from 0x5a4842 to 0x5ac0b8, (0x7876 bytes) 2024-09-13T09:18:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:3057 read 0x22ff784 thru 0x230ce22 (0xd69f bytes) 2024-09-13T09:18:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:3058 read 0x861a5 thru 0x94d84 (0xebe0 bytes) 2024-09-13T09:18:20.076 INFO:teuthology.orchestra.run.smithi032.stdout:3060 read 0x35a68a6 thru 0x35af412 (0x8b6d bytes) 2024-09-13T09:18:20.076 INFO:teuthology.orchestra.run.smithi032.stdout:3063 read 0x217b48b thru 0x21827e5 (0x735b bytes) 2024-09-13T09:18:20.077 INFO:teuthology.orchestra.run.smithi032.stdout:3065 punch from 0xcc9f8b2 to 0xccad34b, (0xda99 bytes) 2024-09-13T09:18:20.077 INFO:teuthology.orchestra.run.smithi032.stdout:3066 punch from 0x30229a3 to 0x302b94e, (0x8fab bytes) 2024-09-13T09:18:20.077 INFO:teuthology.orchestra.run.smithi032.stdout:3067 punch from 0xbaddb39 to 0xbaebaea, (0xdfb1 bytes) 2024-09-13T09:18:20.077 INFO:teuthology.orchestra.run.smithi032.stdout:3068 write 0x919b198 thru 0x919c0e7 (0xf50 bytes) 2024-09-13T09:18:20.082 INFO:teuthology.orchestra.run.smithi032.stdout:3069 writesame 0x484a58f thru 0x484c301 data_size 0x7(0x1d73 bytes) 2024-09-13T09:18:20.098 INFO:teuthology.orchestra.run.smithi032.stdout:3070 trunc from 0xe84fcc6 to 0x8afa77c 2024-09-13T09:18:20.109 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:18:20.111 INFO:teuthology.orchestra.run.smithi032.stdout:3071 write 0x2ce0d21 thru 0x2ce8d24 (0x8004 bytes) 2024-09-13T09:18:20.117 INFO:teuthology.orchestra.run.smithi032.stdout:3073 trunc from 0x8afa77c to 0xb9bfb00 2024-09-13T09:18:20.138 INFO:teuthology.orchestra.run.smithi032.stdout:3076 trunc from 0xb9bfb00 to 0x2cba5fb 2024-09-13T09:18:20.166 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:20.162+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:20.191 INFO:teuthology.orchestra.run.smithi032.stdout:3079 write 0xc46738e thru 0xc46ae57 (0x3aca bytes) 2024-09-13T09:18:20.198 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:20.198 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:20.198 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:20.199 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:20.199 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:20.214 INFO:teuthology.orchestra.run.smithi032.stdout:3080 write 0xd9849d1 thru 0xd98658f (0x1bbf bytes) 2024-09-13T09:18:20.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:20.221 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:20.221 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:20.221 INFO:teuthology.orchestra.run.smithi032.stdout:3084 read 0x6344589 thru 0x634cd01 (0x8779 bytes) 2024-09-13T09:18:20.222 INFO:teuthology.orchestra.run.smithi032.stdout:3086 read 0xac1f1af thru 0xac25a6f (0x68c1 bytes) 2024-09-13T09:18:20.234 INFO:teuthology.orchestra.run.smithi032.stdout:3087 trunc from 0xd986590 to 0x6151986 2024-09-13T09:18:20.267 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:20.262+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:20.267 INFO:teuthology.orchestra.run.smithi032.stdout:3088 trunc from 0x6151986 to 0xd8ef1ba 2024-09-13T09:18:20.278 INFO:teuthology.orchestra.run.smithi032.stdout:3089 read 0xb46ac9f thru 0xb47026b (0x55cd bytes) 2024-09-13T09:18:20.279 INFO:teuthology.orchestra.run.smithi032.stdout:3090 writesame 0x2b77d04 thru 0x2b7c0d7 data_size 0x6(0x43d4 bytes) 2024-09-13T09:18:20.283 INFO:teuthology.orchestra.run.smithi032.stdout:3091 punch from 0xc68e042 to 0xc69074f, (0x270d bytes) 2024-09-13T09:18:20.294 INFO:teuthology.orchestra.run.smithi032.stdout:3092 trunc from 0xd8ef1ba to 0x728019c 2024-09-13T09:18:20.306 INFO:teuthology.orchestra.run.smithi032.stdout:3093 punch from 0xdc75ce to 0xdd3639, (0xc06b bytes) 2024-09-13T09:18:20.328 INFO:teuthology.orchestra.run.smithi032.stdout:3094 writesame 0xee0e434 thru 0xee1cfc8 data_size 0x3(0xeb95 bytes) 2024-09-13T09:18:20.349 INFO:teuthology.orchestra.run.smithi032.stdout:3096 trunc from 0xee1cfc9 to 0x3639aa8 2024-09-13T09:18:20.355 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:20.355 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:20.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:20.355 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:20.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.355 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33548 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.355 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:18:20.356 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.362727279, 2024-09-13T09:18:20.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025760904000000001, 2024-09-13T09:18:20.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33548 2024-09-13T09:18:20.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.356 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:18:20.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:18:20.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:18:20.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990124+0000", 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990130+0000", 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990135+0000", 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007826+0000", 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017691018999999999 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007896+0000", 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0059000000000001e-05 2024-09-13T09:18:20.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.015883+0000", 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079875269999999995 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33750 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993520+0000", 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3593294610000002, 2024-09-13T09:18:20.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023633346999999999, 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33750 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993520+0000", 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993520+0000", 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993520+0000", 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993522+0000", 2024-09-13T09:18:20.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.939e-06 2024-09-13T09:18:20.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993528+0000", 2024-09-13T09:18:20.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0480000000000004e-06 2024-09-13T09:18:20.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993538+0000", 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8949999999999993e-06 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017046+0000", 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023508322000000002 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017108+0000", 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2250999999999997e-05 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017154+0000", 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5201000000000003e-05 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33751 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1593344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.359307625, 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023779101, 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33751 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993548+0000", 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.776e-06 2024-09-13T09:18:20.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993555+0000", 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993560+0000", 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017188+0000", 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023627984000000001 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017271+0000", 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2798000000000002e-05 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017321+0000", 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0557999999999999e-05 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33752 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3592855049999999, 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023925173000000001, 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33752 2024-09-13T09:18:20.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993566+0000", 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993573+0000", 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.759e-06 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993578+0000", 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8339999999999998e-06 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017349+0000", 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023770959000000001 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017423+0000", 2024-09-13T09:18:20.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3652000000000006e-05 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017489+0000", 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6748999999999994e-05 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33753 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1650688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3592670230000001, 2024-09-13T09:18:20.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024066349000000001, 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33753 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993582+0000", 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993585+0000", 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993591+0000", 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993595+0000", 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017515+0000", 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023920117000000001 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017583+0000", 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8485999999999994e-05 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017649+0000", 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:18:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33754 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3592507290000002, 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02420106, 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33754 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993601+0000", 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993607+0000", 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9669999999999998e-06 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993611+0000", 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017669+0000", 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024057683999999999 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017742+0000", 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3397000000000002e-05 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017800+0000", 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7711999999999999e-05 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33755 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1708032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3592343100000002, 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024346923999999999, 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33755 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993617+0000", 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993624+0000", 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0469999999999999e-06 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993628+0000", 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:18:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017830+0000", 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024201879999999999 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017897+0000", 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7096999999999998e-05 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017962+0000", 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5455000000000003e-05 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33756 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.359217584, 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024485512000000001, 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33756 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993635+0000", 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993641+0000", 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993645+0000", 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2520000000000001e-06 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017979+0000", 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024333950999999999 2024-09-13T09:18:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018045+0000", 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5843e-05 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018118+0000", 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2417999999999997e-05 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33757 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1765376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3592001859999998, 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024605578999999999, 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33757 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993649+0000", 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993651+0000", 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.841e-06 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993657+0000", 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993662+0000", 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018139+0000", 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024476748 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018196+0000", 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7188000000000002e-05 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018255+0000", 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9045999999999997e-05 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33758 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3591835529999998, 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024736898, 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33758 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993668+0000", 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993674+0000", 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993679+0000", 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.515e-06 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018284+0000", 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024604816000000002 2024-09-13T09:18:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018356+0000", 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2782999999999998e-05 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018403+0000", 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6726e-05 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33759 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1822720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3591669469999998, 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873635000000002, 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33759 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993682+0000", 2024-09-13T09:18:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993685+0000", 2024-09-13T09:18:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:18:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993691+0000", 2024-09-13T09:18:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1840000000000004e-06 2024-09-13T09:18:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993696+0000", 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018429+0000", 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024733330000000001 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018494+0000", 2024-09-13T09:18:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5233000000000004e-05 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018556+0000", 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2125999999999994e-05 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33760 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:18:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3591500920000001, 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025058980000000002, 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33760 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993699+0000", 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993702+0000", 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.632e-06 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993708+0000", 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2280000000000003e-06 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993713+0000", 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018586+0000", 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873572 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018645+0000", 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9098000000000002e-05 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018759+0000", 2024-09-13T09:18:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113224 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33761 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1880064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3591333950000002, 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025229076, 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33761 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993718+0000", 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.092e-06 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993724+0000", 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0839999999999997e-06 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993729+0000", 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018790+0000", 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025061039 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018871+0000", 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0885999999999998e-05 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018945+0000", 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4773999999999999e-05 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33762 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.359116921, 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025559609, 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33762 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993732+0000", 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993735+0000", 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993740+0000", 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993745+0000", 2024-09-13T09:18:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5290000000000002e-06 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019043+0000", 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025298355000000002 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019134+0000", 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0781999999999999e-05 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019292+0000", 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015829 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33763 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.35910098, 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025732657999999999, 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33763 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:18:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993748+0000", 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993755+0000", 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993762+0000", 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993767+0000", 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7230000000000004e-06 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019322+0000", 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025555564999999999 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019402+0000", 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9433999999999999e-05 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019481+0000", 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9739000000000003e-05 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33764 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3590786060000002, 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025908526000000001, 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33764 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993773+0000", 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993779+0000", 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9610000000000002e-06 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993783+0000", 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.459e-06 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019513+0000", 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025729399 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019609+0000", 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6504999999999998e-05 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019680+0000", 2024-09-13T09:18:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0393000000000002e-05 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33765 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1994752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3590623709999998, 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026051009999999999, 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33765 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993789+0000", 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993795+0000", 2024-09-13T09:18:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.088e-06 2024-09-13T09:18:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993799+0000", 2024-09-13T09:18:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:18:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019704+0000", 2024-09-13T09:18:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025904311999999999 2024-09-13T09:18:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019766+0000", 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2829000000000004e-05 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019838+0000", 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1935999999999997e-05 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33766 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3590463160000001, 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026238971999999999, 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33766 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:18:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:18:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:18:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993805+0000", 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993811+0000", 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9000000000000003e-06 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993816+0000", 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.527e-06 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019877+0000", 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026061337 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019960+0000", 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3245999999999995e-05 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020042+0000", 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2053999999999997e-05 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33767 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.359029906, 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026407013999999999, 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33767 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993819+0000", 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993822+0000", 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993828+0000", 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993832+0000", 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.493e-06 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020072+0000", 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026239519999999999 2024-09-13T09:18:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020151+0000", 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9278999999999998e-05 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020227+0000", 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5965999999999997e-05 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33768 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.3590114529999999, 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026564424, 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33768 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993840+0000", 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.996e-06 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993846+0000", 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993858+0000", 2024-09-13T09:18:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2214e-05 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020253+0000", 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026394926999999999 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020329+0000", 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5193999999999999e-05 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020403+0000", 2024-09-13T09:18:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4145000000000002e-05 2024-09-13T09:18:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.447 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:20.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:20.449 INFO:teuthology.orchestra.run.smithi032.stdout:3097 writesame 0xe6cd11b thru 0xe6d6742 data_size 0x3e(0x9628 bytes) 2024-09-13T09:18:20.449 INFO:teuthology.orchestra.run.smithi032.stdout:3099 trunc from 0xe6d6743 to 0x3883b46 2024-09-13T09:18:20.449 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:20.447+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:20.454 INFO:teuthology.orchestra.run.smithi032.stdout:3100 writesame 0xd54a8ce thru 0xd5568a2 data_size 0x1(0xbfd5 bytes) 2024-09-13T09:18:20.454 INFO:teuthology.orchestra.run.smithi032.stdout:3101 write 0x91f32cb thru 0x91ff3a0 (0xc0d6 bytes) 2024-09-13T09:18:20.464 INFO:teuthology.orchestra.run.smithi032.stdout:3102 writesame 0xa1f275c thru 0xa1f669d data_size 0x3(0x3f42 bytes) 2024-09-13T09:18:20.474 INFO:teuthology.orchestra.run.smithi032.stdout:3104 read 0x421ec8c thru 0x422ae3b (0xc1b0 bytes) 2024-09-13T09:18:20.475 INFO:teuthology.orchestra.run.smithi032.stdout:3105 write 0x11425ea thru 0x114b8c7 (0x92de bytes) 2024-09-13T09:18:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:3106 write 0xd98ef82 thru 0xd997eea (0x8f69 bytes) 2024-09-13T09:18:20.505 INFO:teuthology.orchestra.run.smithi032.stdout:3108 writesame 0xbeba429 thru 0xbebb248 data_size 0x10(0xe20 bytes) 2024-09-13T09:18:20.510 INFO:teuthology.orchestra.run.smithi032.stdout:3109 write 0xc768833 thru 0xc76e381 (0x5b4f bytes) 2024-09-13T09:18:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:3110 write 0x3477023 thru 0x347c10c (0x50ea bytes) 2024-09-13T09:18:20.526 INFO:teuthology.orchestra.run.smithi032.stdout:3111 read 0x5cbecc2 thru 0x5cc11cc (0x250b bytes) 2024-09-13T09:18:20.533 INFO:teuthology.orchestra.run.smithi032.stdout:3112 trunc from 0xd997eeb to 0x8f26f6b 2024-09-13T09:18:20.548 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:20.547+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:20.561 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:20.561 INFO:teuthology.orchestra.run.smithi032.stdout:3114 punch from 0x453688d to 0x45375c0, (0xd33 bytes) 2024-09-13T09:18:20.561 INFO:teuthology.orchestra.run.smithi032.stdout:3115 write 0x304d65b thru 0x305bc8e (0xe634 bytes) 2024-09-13T09:18:20.571 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:20.571 INFO:teuthology.orchestra.run.smithi032.stdout:3117 read 0x1f1e9b0 thru 0x1f2b510 (0xcb61 bytes) 2024-09-13T09:18:20.572 INFO:teuthology.orchestra.run.smithi032.stdout:3118 writesame 0x525e4f9 thru 0x525e5de data_size 0x5(0xe6 bytes) 2024-09-13T09:18:20.576 INFO:teuthology.orchestra.run.smithi032.stdout:3119 read 0x1d1a07a thru 0x1d20f8a (0x6f11 bytes) 2024-09-13T09:18:20.597 INFO:teuthology.orchestra.run.smithi032.stdout:3120 write 0xe93b148 thru 0xe949109 (0xdfc2 bytes) 2024-09-13T09:18:20.597 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:20.597 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:20.597 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:20.598 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:20.608 INFO:teuthology.orchestra.run.smithi032.stdout:3122 writesame 0x40ea6b8 thru 0x40ee8f8 data_size 0x1(0x4241 bytes) 2024-09-13T09:18:20.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:20.618 INFO:teuthology.orchestra.run.smithi032.stdout:3124 read 0x1ca8e42 thru 0x1cafd3d (0x6efc bytes) 2024-09-13T09:18:20.618 INFO:teuthology.orchestra.run.smithi032.stdout:3125 read 0x31335e5 thru 0x313f1db (0xbbf7 bytes) 2024-09-13T09:18:20.620 INFO:teuthology.orchestra.run.smithi032.stdout:3126 writesame 0x6eaa3f9 thru 0x6eb4cc6 data_size 0x2(0xa8ce bytes) 2024-09-13T09:18:20.635 INFO:teuthology.orchestra.run.smithi032.stdout:3127 read 0x31e2246 thru 0x31e2f8c (0xd47 bytes) 2024-09-13T09:18:20.636 INFO:teuthology.orchestra.run.smithi032.stdout:3128 writesame 0x79dada2 thru 0x79e73b2 data_size 0x1(0xc611 bytes) 2024-09-13T09:18:20.649 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:20.647+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:20.679 INFO:teuthology.orchestra.run.smithi032.stdout:3129 trunc from 0xe94910a to 0xa26ea7f 2024-09-13T09:18:20.704 INFO:teuthology.orchestra.run.smithi032.stdout:3131 read 0x80d0c4 thru 0x815273 (0x81b0 bytes) 2024-09-13T09:18:20.705 INFO:teuthology.orchestra.run.smithi032.stdout:3132 write 0x17b9947 thru 0x17bc707 (0x2dc1 bytes) 2024-09-13T09:18:20.716 INFO:teuthology.orchestra.run.smithi032.stdout:3133 read 0x5b54fde thru 0x5b59144 (0x4167 bytes) 2024-09-13T09:18:20.716 INFO:teuthology.orchestra.run.smithi032.stdout:3134 read 0x5bd8f26 thru 0x5be3969 (0xaa44 bytes) 2024-09-13T09:18:20.716 INFO:teuthology.orchestra.run.smithi032.stdout:3135 write 0x611613c thru 0x611b4a2 (0x5367 bytes) 2024-09-13T09:18:20.723 INFO:teuthology.orchestra.run.smithi032.stdout:3136 read 0x4fb9de4 thru 0x4fc04e7 (0x6704 bytes) 2024-09-13T09:18:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:20.745 INFO:teuthology.orchestra.run.smithi032.stdout:3140 writesame 0xe5d562d thru 0xe5e095f data_size 0x19(0xb333 bytes) 2024-09-13T09:18:20.746 INFO:teuthology.orchestra.run.smithi032.stdout:3141 writesame 0x37e6cf5 thru 0x37eb310 data_size 0xe(0x461c bytes) 2024-09-13T09:18:20.750 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:20.747+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:20.756 INFO:teuthology.orchestra.run.smithi032.stdout:3142 read 0x7beda2a thru 0x7bf8259 (0xa830 bytes) 2024-09-13T09:18:20.757 INFO:teuthology.orchestra.run.smithi032.stdout:3143 punch from 0x5d3cab1 to 0x5d4a2fb, (0xd84a bytes) 2024-09-13T09:18:20.757 INFO:teuthology.orchestra.run.smithi032.stdout:3144 write 0x1c23cd3 thru 0x1c24c75 (0xfa3 bytes) 2024-09-13T09:18:20.759 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:20.767 INFO:teuthology.orchestra.run.smithi032.stdout:3145 read 0x6104198 thru 0x610f3c5 (0xb22e bytes) 2024-09-13T09:18:20.767 INFO:teuthology.orchestra.run.smithi032.stdout:3146 write 0x492ccd7 thru 0x493c221 (0xf54b bytes) 2024-09-13T09:18:20.778 INFO:teuthology.orchestra.run.smithi032.stdout:3147 write 0x66dd35f thru 0x66deec4 (0x1b66 bytes) 2024-09-13T09:18:20.785 INFO:teuthology.orchestra.run.smithi032.stdout:3150 read 0x60a5898 thru 0x60b06e1 (0xae4a bytes) 2024-09-13T09:18:20.792 INFO:teuthology.orchestra.run.smithi032.stdout:3151 trunc from 0xe5e0960 to 0x9ba2cfe 2024-09-13T09:18:20.806 INFO:teuthology.orchestra.run.smithi032.stdout:3152 read 0x9704e47 thru 0x970a928 (0x5ae2 bytes) 2024-09-13T09:18:20.806 INFO:teuthology.orchestra.run.smithi032.stdout:3153 write 0x4e0fc82 thru 0x4e193f4 (0x9773 bytes) 2024-09-13T09:18:20.835 INFO:teuthology.orchestra.run.smithi032.stdout:3155 write 0xb798d5f thru 0xb79e0e3 (0x5385 bytes) 2024-09-13T09:18:20.844 INFO:teuthology.orchestra.run.smithi032.stdout:3157 read 0x8511402 thru 0x851bba6 (0xa7a5 bytes) 2024-09-13T09:18:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:3158 read 0x78c1d8a thru 0x78d0675 (0xe8ec bytes) 2024-09-13T09:18:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:3159 writesame 0x539b7d8 thru 0x53a1737 data_size 0x6d(0x5f60 bytes) 2024-09-13T09:18:20.851 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:20.846+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:20.871 INFO:teuthology.orchestra.run.smithi032.stdout:3160 writesame 0xc2f31ce thru 0xc2fc700 data_size 0x1(0x9533 bytes) 2024-09-13T09:18:20.872 INFO:teuthology.orchestra.run.smithi032.stdout:3162 punch from 0x25f7bc9 to 0x2603c5a, (0xc091 bytes) 2024-09-13T09:18:20.872 INFO:teuthology.orchestra.run.smithi032.stdout:3163 writesame 0x1689e4a thru 0x168f090 data_size 0x15(0x5247 bytes) 2024-09-13T09:18:20.882 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:20.882 INFO:teuthology.orchestra.run.smithi032.stdout:3167 write 0x8a83d29 thru 0x8a91aa0 (0xdd78 bytes) 2024-09-13T09:18:20.897 INFO:teuthology.orchestra.run.smithi032.stdout:3168 read 0x41040e9 thru 0x41082f2 (0x420a bytes) 2024-09-13T09:18:20.898 INFO:teuthology.orchestra.run.smithi032.stdout:3169 read 0x607f1e thru 0x60b9c4 (0x3aa7 bytes) 2024-09-13T09:18:20.901 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.361478263000002, 2024-09-13T09:18:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:18:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:18:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:18:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:18:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:18:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:18:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:18:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:18:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:18:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:18:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:18:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:18:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:18:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:18:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:18:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:18:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:18:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:18:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:18:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:18:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:18:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:18:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:18:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32899 3.18 3:1fefe8db:::rbd_data.1185833655de.000000000000001e:16 [read 704512~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9919312150000001, 2024-09-13T09:18:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046124828, 2024-09-13T09:18:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32899 2024-09-13T09:18:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905907+0000", 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905909+0000", 2024-09-13T09:18:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905919+0000", 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0520000000000007e-06 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905923+0000", 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.951907+0000", 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.045983221999999997 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.951969+0000", 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2442999999999998e-05 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952032+0000", 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999998e-05 2024-09-13T09:18:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9919112779999999, 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:18:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:18:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:18:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:18:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:18:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:18:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:18:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9918939570000003, 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:18:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9918770160000001, 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:18:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:18:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:18:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:18:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:18:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:18:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:18:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:18:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:18:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:18:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:18:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9918597880000002, 2024-09-13T09:18:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:18:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:18:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:18:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:18:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:18:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:18:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:18:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:18:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:18:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:18:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:18:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:18:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9918233669999998, 2024-09-13T09:18:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:18:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:18:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:18:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:18:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:18:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:18:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:18:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:18:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:18:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:18:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:18:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.991769905, 2024-09-13T09:18:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:18:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:18:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:18:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:18:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:18:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:18:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:18:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:18:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:18:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:18:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:18:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:18:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:18:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9917478850000001, 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:18:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:18:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:18:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:18:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:18:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:18:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9917232499999997, 2024-09-13T09:18:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:18:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:18:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9917047270000001, 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:18:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:18:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9916868559999998, 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:18:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9916695259999999, 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:18:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:18:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9916520559999999, 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:18:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:18:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:18:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:18:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:18:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9916351900000002, 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:18:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:18:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:18:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:18:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:18:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:18:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:18:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:18:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:18:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9916174030000002, 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:18:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:18:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:18:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:18:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:18:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:18:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:18:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:18:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:18:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:18:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.991573093, 2024-09-13T09:18:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:18:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:18:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.7584426049999999, 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:18:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:18:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:18:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:18:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:18:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:18:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:18:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.7584257719999998, 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:18:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:18:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:18:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:18:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:18:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:18:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:18:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:18:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:18:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.7584094139999999, 2024-09-13T09:18:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:18:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:18:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:18:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:18:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:21.006 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:21.006 INFO:teuthology.orchestra.run.smithi032.stdout:3170 read 0xb9def37 thru 0xb9e3398 (0x4462 bytes) 2024-09-13T09:18:21.006 INFO:teuthology.orchestra.run.smithi032.stdout:3171 write 0x4b73a7d thru 0x4b7b89d (0x7e21 bytes) 2024-09-13T09:18:21.006 INFO:teuthology.orchestra.run.smithi032.stdout:3172 read 0x42b8c57 thru 0x42c88b7 (0xfc61 bytes) 2024-09-13T09:18:21.006 INFO:teuthology.orchestra.run.smithi032.stdout:3173 write 0x308e127 thru 0x309a675 (0xc54f bytes) 2024-09-13T09:18:21.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:21.009 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:21.009 INFO:teuthology.orchestra.run.smithi032.stdout:3177 punch from 0x874c3f3 to 0x875373f, (0x734c bytes) 2024-09-13T09:18:21.009 INFO:teuthology.orchestra.run.smithi032.stdout:3178 read 0x844d1bf thru 0x8451402 (0x4244 bytes) 2024-09-13T09:18:21.009 INFO:teuthology.orchestra.run.smithi032.stdout:3179 trunc from 0xc2fc701 to 0xe7d5534 2024-09-13T09:18:21.009 INFO:teuthology.orchestra.run.smithi032.stdout:3181 write 0x61d9fea thru 0x61dc042 (0x2059 bytes) 2024-09-13T09:18:21.009 INFO:teuthology.orchestra.run.smithi032.stdout:3182 read 0x70b1336 thru 0x70bd1fd (0xbec8 bytes) 2024-09-13T09:18:21.010 INFO:teuthology.orchestra.run.smithi032.stdout:3183 punch from 0x42d1b50 to 0x42d1fb2, (0x462 bytes) 2024-09-13T09:18:21.010 INFO:teuthology.orchestra.run.smithi032.stdout:3184 trunc from 0xe7d5534 to 0x3552b43 2024-09-13T09:18:21.010 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:18:21.010 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:18:21.010 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-13T09:18:21.010 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:18:21.010 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:18:21.011 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-13T09:18:21.019 INFO:teuthology.orchestra.run.smithi032.stdout:3186 punch from 0x30a71a0 to 0x30b1aeb, (0xa94b bytes) 2024-09-13T09:18:21.019 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:21.019 INFO:teuthology.orchestra.run.smithi032.stdout:3189 read 0x1624a90 thru 0x162b482 (0x69f3 bytes) 2024-09-13T09:18:21.019 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:21.019+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:21.020 INFO:teuthology.orchestra.run.smithi032.stdout:3190 punch from 0x11fe65d to 0x1208ddc, (0xa77f bytes) 2024-09-13T09:18:21.020 INFO:teuthology.orchestra.run.smithi032.stdout:3191 write 0xed6c4f thru 0xedf292 (0x8644 bytes) 2024-09-13T09:18:21.085 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:21.085 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:21.088 INFO:teuthology.orchestra.run.smithi032.stdout:3194 trunc from 0x3552b43 to 0x4e49f42 2024-09-13T09:18:21.099 INFO:teuthology.orchestra.run.smithi032.stdout:3196 read 0x3323d14 thru 0x332fc25 (0xbf12 bytes) 2024-09-13T09:18:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:18:21.112 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:21.111+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:21.118 INFO:teuthology.orchestra.run.smithi032.stdout:3197 write 0x712dfef thru 0x713c005 (0xe017 bytes) 2024-09-13T09:18:21.152 INFO:teuthology.orchestra.run.smithi032.stdout:3198 write 0xe551d16 thru 0xe554264 (0x254f bytes) 2024-09-13T09:18:21.158 INFO:teuthology.orchestra.run.smithi032.stdout:3199 write 0x4acdb18 thru 0x4ad17f2 (0x3cdb bytes) 2024-09-13T09:18:21.179 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:21.179 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:18:21.180 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:21.186 INFO:teuthology.orchestra.run.smithi032.stdout:3200 trunc from 0xe554265 to 0x3e8a411 2024-09-13T09:18:21.186 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:21.186 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:21.186 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:21.186 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:21.205 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:21.213 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:21.211+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:21.224 INFO:teuthology.orchestra.run.smithi032.stdout:3201 write 0x7a6ddf3 thru 0x7a73e6f (0x607d bytes) 2024-09-13T09:18:21.251 INFO:teuthology.orchestra.run.smithi032.stdout:3203 write 0xb4facd9 thru 0xb5017b1 (0x6ad9 bytes) 2024-09-13T09:18:21.264 INFO:teuthology.orchestra.run.smithi032.stdout:3204 read 0x588607d thru 0x589529e (0xf222 bytes) 2024-09-13T09:18:21.264 INFO:teuthology.orchestra.run.smithi032.stdout:3205 read 0x4e07bfe thru 0x4e0f1ba (0x75bd bytes) 2024-09-13T09:18:21.276 INFO:teuthology.orchestra.run.smithi032.stdout:3207 trunc from 0xb5017b2 to 0x72655ce 2024-09-13T09:18:21.297 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:21.314 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:21.310+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:21.319 INFO:teuthology.orchestra.run.smithi032.stdout:3209 write 0xb28588c thru 0xb28676c (0xee1 bytes) 2024-09-13T09:18:21.327 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:21.347 INFO:teuthology.orchestra.run.smithi032.stdout:3211 write 0xd8fc219 thru 0xd8feeba (0x2ca2 bytes) 2024-09-13T09:18:21.354 INFO:teuthology.orchestra.run.smithi032.stdout:3212 write 0x9778654 thru 0x978057f (0x7f2c bytes) 2024-09-13T09:18:21.367 INFO:teuthology.orchestra.run.smithi032.stdout:3213 read 0x1d0b962 thru 0x1d126bc (0x6d5b bytes) 2024-09-13T09:18:21.368 INFO:teuthology.orchestra.run.smithi032.stdout:3215 read 0x3db6c3a thru 0x3db9a81 (0x2e48 bytes) 2024-09-13T09:18:21.368 INFO:teuthology.orchestra.run.smithi032.stdout:3216 read 0xb438a7 thru 0xb508d6 (0xd030 bytes) 2024-09-13T09:18:21.370 INFO:teuthology.orchestra.run.smithi032.stdout:3219 punch from 0xcf14837 to 0xcf18627, (0x3df0 bytes) 2024-09-13T09:18:21.370 INFO:teuthology.orchestra.run.smithi032.stdout:3220 writesame 0xd445167 thru 0xd454339 data_size 0x1(0xf1d3 bytes) 2024-09-13T09:18:21.376 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:21.376 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:21.376 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:21.376 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:21.376 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:21.394 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:21.413 INFO:teuthology.orchestra.run.smithi032.stdout:3221 punch from 0x7d3c584 to 0x7d3debc, (0x1938 bytes) 2024-09-13T09:18:21.413 INFO:teuthology.orchestra.run.smithi032.stdout:3223 punch from 0x8dc515e to 0x8dc7c39, (0x2adb bytes) 2024-09-13T09:18:21.413 INFO:teuthology.orchestra.run.smithi032.stdout:3224 write 0x7a15656 thru 0x7a1600c (0x9b7 bytes) 2024-09-13T09:18:21.414 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:21.415+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:21.419 INFO:teuthology.orchestra.run.smithi032.stdout:3225 read 0x16a8b32 thru 0x16aa6cd (0x1b9c bytes) 2024-09-13T09:18:21.420 INFO:teuthology.orchestra.run.smithi032.stdout:3227 punch from 0x6cc6d05 to 0x6ccecad, (0x7fa8 bytes) 2024-09-13T09:18:21.420 INFO:teuthology.orchestra.run.smithi032.stdout:3228 write 0x6e6a087 thru 0x6e777c6 (0xd740 bytes) 2024-09-13T09:18:21.433 INFO:teuthology.orchestra.run.smithi032.stdout:3229 read 0xba9cdf1 thru 0xbaaa090 (0xd2a0 bytes) 2024-09-13T09:18:21.434 INFO:teuthology.orchestra.run.smithi032.stdout:3230 read 0xb072d57 thru 0xb080873 (0xdb1d bytes) 2024-09-13T09:18:21.434 INFO:teuthology.orchestra.run.smithi032.stdout:3231 punch from 0x9ce2ff7 to 0x9ceeb0b, (0xbb14 bytes) 2024-09-13T09:18:21.448 INFO:teuthology.orchestra.run.smithi032.stdout:3232 trunc from 0xd8feebb to 0x8698c73 2024-09-13T09:18:21.474 INFO:teuthology.orchestra.run.smithi032.stdout:3235 punch from 0x170e70a to 0x17199d0, (0xb2c6 bytes) 2024-09-13T09:18:21.474 INFO:teuthology.orchestra.run.smithi032.stdout:3236 read 0x3d56e37 thru 0x3d608bc (0x9a86 bytes) 2024-09-13T09:18:21.474 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:21.475 INFO:teuthology.orchestra.run.smithi032.stdout:3238 punch from 0x219110a to 0x219a260, (0x9156 bytes) 2024-09-13T09:18:21.475 INFO:teuthology.orchestra.run.smithi032.stdout:3239 clone 21 order 23 su 262144 sc 7 2024-09-13T09:18:21.515 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:21.515+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:21.568 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:21.568 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:21.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4388 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.579400872000001, 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069349509000000004, 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4388 2024-09-13T09:18:21.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985769+0000", 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985775+0000", 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5230000000000001e-06 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985794+0000", 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8425999999999999e-05 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985809+0000", 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.486e-05 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.054965+0000", 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069156463000000001 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055036+0000", 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0780999999999999e-05 2024-09-13T09:18:21.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055120+0000", 2024-09-13T09:18:21.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3486000000000007e-05 2024-09-13T09:18:21.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.573 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.573 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4389 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.573 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:21.573 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.579350349000002, 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069473833999999998, 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4389 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985820+0000", 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985859+0000", 2024-09-13T09:18:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9279999999999997e-05 2024-09-13T09:18:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985881+0000", 2024-09-13T09:18:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2116999999999998e-05 2024-09-13T09:18:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985896+0000", 2024-09-13T09:18:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4698999999999999e-05 2024-09-13T09:18:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055152+0000", 2024-09-13T09:18:21.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069256568000000004 2024-09-13T09:18:21.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055220+0000", 2024-09-13T09:18:21.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8255000000000003e-05 2024-09-13T09:18:21.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055294+0000", 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3858999999999999e-05 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4390 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.579254736999999, 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069563635999999998, 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4390 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985915+0000", 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985922+0000", 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:18:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985941+0000", 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8260000000000001e-05 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985955+0000", 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4409999999999999e-05 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055325+0000", 2024-09-13T09:18:21.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069370290000000001 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055395+0000", 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0158000000000002e-05 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055480+0000", 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4340999999999998e-05 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.579203669999998, 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:18:21.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:18:21.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.579154934000002, 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:18:21.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:18:21.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.579087264999998, 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.592 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:18:21.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:18:21.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:18:21.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:18:21.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:18:21.596 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.596 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.596 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.597 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.597 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:21.597 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578947055999997, 2024-09-13T09:18:21.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:18:21.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.597 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:18:21.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.597 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:18:21.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578856190000003, 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:18:21.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.602 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.602 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.602 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.602 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:18:21.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.602 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:21.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:21.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:18:21.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:18:21.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578781329999998, 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.607 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:18:21.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.607 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:21.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:18:21.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:18:21.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:18:21.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:18:21.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:18:21.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:18:21.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578710153999999, 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:18:21.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:18:21.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:18:21.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:18:21.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:18:21.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:18:21.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:18:21.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:18:21.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.616 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.617 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.617 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:21.617 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578661101000002, 2024-09-13T09:18:21.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:18:21.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:18:21.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.617 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:18:21.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:18:21.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:18:21.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:18:21.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:18:21.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:18:21.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:18:21.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:18:21.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:18:21.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:18:21.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:18:21.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:18:21.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:18:21.621 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.621 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.621 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.621 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578599365000002, 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:18:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:18:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:18:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:18:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:18:21.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:18:21.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:18:21.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:18:21.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:18:21.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578549695, 2024-09-13T09:18:21.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:18:21.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:18:21.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.627 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:18:21.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:18:21.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:18:21.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:18:21.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:18:21.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:18:21.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578501176000003, 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:18:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.632 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.632 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.632 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.632 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:18:21.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.632 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:21.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:18:21.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:18:21.636 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.636 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.636 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.636 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.636 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:21.636 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578452779999999, 2024-09-13T09:18:21.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:18:21.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.636 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:21.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:18:21.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:18:21.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:18:21.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:18:21.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:18:21.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:18:21.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:18:21.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:18:21.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578405492999998, 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:18:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:18:21.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:18:21.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:18:21.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:21.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:18:21.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:18:21.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:18:21.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:18:21.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:18:21.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:18:21.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578322090999997, 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.646 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:18:21.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:18:21.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:18:21.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:18:21.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:18:21.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.650 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578248365, 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:21.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:21.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:18:21.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:18:21.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:18:21.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:18:21.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:18:21.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:18:21.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:18:21.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.655 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.655 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.655 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:21.655 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578180242000002, 2024-09-13T09:18:21.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:18:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:18:21.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:18:21.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:18:21.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.660 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.660 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:21.660 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:21.660 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.578129775000001, 2024-09-13T09:18:21.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:18:21.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:21.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:21.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:21.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:18:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:18:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:18:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:18:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:21.665 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:21.666 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:21.666 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:21.662+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:21.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:18:21.767 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:21.762+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:21.841 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:21.842 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:21.842 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:21.842 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:21.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:21.868 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:21.867+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:21.916 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:21.916 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:21.934 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:21.934 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:21.950 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:21.967 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:21.969 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:21.962+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:21.987 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:22.007 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:22.019 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:22.019 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:22.029 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:22.034 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:22.050 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:22.051 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:22.077 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:22.074+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:22.171 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:22.167+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:22.210 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:22.210 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:22.225 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:22.235 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone20 from 0x8698c73 (overlap 0x1e8b765) to 0x1253ad6 2024-09-13T09:18:22.271 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:22.271+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:22.364 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:22.364 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:22.371 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:22.371+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:22.379 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:22.426 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-13T09:18:22.472 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:22.466+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:22.500 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:22.500 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:22.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:22.602 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:22.598+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:22.603 INFO:teuthology.orchestra.run.smithi032.stdout:3240 write 0xde9923e thru 0xde9fd27 (0x6aea bytes) 2024-09-13T09:18:22.610 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:22.610 INFO:teuthology.orchestra.run.smithi032.stdout:3242 write 0x64b1fdf thru 0x64bc734 (0xa756 bytes) 2024-09-13T09:18:22.622 INFO:teuthology.orchestra.run.smithi032.stdout:3243 read 0x687e79c thru 0x688dc87 (0xf4ec bytes) 2024-09-13T09:18:22.622 INFO:teuthology.orchestra.run.smithi032.stdout:3245 write 0x9366400 thru 0x937274a (0xc34b bytes) 2024-09-13T09:18:22.634 INFO:teuthology.orchestra.run.smithi032.stdout:3247 punch from 0x89b2c05 to 0x89bed50, (0xc14b bytes) 2024-09-13T09:18:22.634 INFO:teuthology.orchestra.run.smithi032.stdout:3248 read 0x5a88828 thru 0x5a9134a (0x8b23 bytes) 2024-09-13T09:18:22.634 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:22.635 INFO:teuthology.orchestra.run.smithi032.stdout:3251 write 0xc6a62c7 thru 0xc6ad20a (0x6f44 bytes) 2024-09-13T09:18:22.657 INFO:teuthology.orchestra.run.smithi032.stdout:3252 write 0xed94649 thru 0xed9e5f7 (0x9faf bytes) 2024-09-13T09:18:22.665 INFO:teuthology.orchestra.run.smithi032.stdout:3254 punch from 0xc9fd1e5 to 0xc9ff3b1, (0x21cc bytes) 2024-09-13T09:18:22.675 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:22.670+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:22.679 INFO:teuthology.orchestra.run.smithi032.stdout:3256 trunc from 0xed9e5f8 to 0x996e8ec 2024-09-13T09:18:22.693 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:22.693 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:22.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:22.736 INFO:teuthology.orchestra.run.smithi032.stdout:3258 write 0x170db46 thru 0x17103a0 (0x285b bytes) 2024-09-13T09:18:22.775 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:22.775+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:22.805 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:22.805 INFO:teuthology.orchestra.run.smithi032.stdout:3261 writesame 0x7517de4 thru 0x7520a72 data_size 0x1(0x8c8f bytes) 2024-09-13T09:18:22.827 INFO:teuthology.orchestra.run.smithi032.stdout:3262 trunc from 0x996e8ec to 0x978ce14 2024-09-13T09:18:22.842 INFO:teuthology.orchestra.run.smithi032.stdout:3263 write 0x8152a5b thru 0x8160f07 (0xe4ad bytes) 2024-09-13T09:18:22.845 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:22.846 INFO:teuthology.orchestra.run.smithi032.stdout:3266 read 0x62c6fc4 thru 0x62d6360 (0xf39d bytes) 2024-09-13T09:18:22.847 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:22.847 INFO:teuthology.orchestra.run.smithi032.stdout:3268 punch from 0x4c6ce64 to 0x4c7c26a, (0xf406 bytes) 2024-09-13T09:18:22.847 INFO:teuthology.orchestra.run.smithi032.stdout:3269 punch from 0x7bce756 to 0x7bdc395, (0xdc3f bytes) 2024-09-13T09:18:22.858 INFO:teuthology.orchestra.run.smithi032.stdout:3270 writesame 0x98f9f45 thru 0x98fe8e5 data_size 0x3(0x49a1 bytes) 2024-09-13T09:18:22.861 INFO:teuthology.orchestra.run.smithi032.stdout:3272 punch from 0x1c4d68 to 0x1cfacb, (0xad63 bytes) 2024-09-13T09:18:22.874 INFO:teuthology.orchestra.run.smithi032.stdout:3274 trunc from 0x98fe8e6 to 0xe20c091 2024-09-13T09:18:22.875 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:22.870+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:22.893 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:22.893 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:22.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:22.908 INFO:teuthology.orchestra.run.smithi032.stdout:3275 trunc from 0xe20c091 to 0x721456e 2024-09-13T09:18:22.924 INFO:teuthology.orchestra.run.smithi032.stdout:3276 punch from 0x501d9d1 to 0x5020584, (0x2bb3 bytes) 2024-09-13T09:18:22.925 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:22.938 INFO:teuthology.orchestra.run.smithi032.stdout:3278 writesame 0x7407bfd thru 0x740dd30 data_size 0x2(0x6134 bytes) 2024-09-13T09:18:22.941 INFO:teuthology.orchestra.run.smithi032.stdout:3280 writesame 0x5ac5568 thru 0x5ace032 data_size 0x1(0x8acb bytes) 2024-09-13T09:18:22.984 INFO:teuthology.orchestra.run.smithi032.stdout:3284 write 0x40a5770 thru 0x40ab7ac (0x603d bytes) 2024-09-13T09:18:22.988 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:22.988 INFO:teuthology.orchestra.run.smithi032.stdout:3287 writesame 0x40d5063 thru 0x40deefc data_size 0x43(0x9e9a bytes) 2024-09-13T09:18:22.997 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:22.994+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:3290 write 0x3118d01 thru 0x311aa40 (0x1d40 bytes) 2024-09-13T09:18:23.077 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:23.075+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:23.100 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:23.100 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:23.111 INFO:teuthology.orchestra.run.smithi032.stdout:3291 write 0xb702937 thru 0xb70a531 (0x7bfb bytes) 2024-09-13T09:18:23.115 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:23.118 INFO:teuthology.orchestra.run.smithi032.stdout:3294 read 0x15b8fd9 thru 0x15be99c (0x59c4 bytes) 2024-09-13T09:18:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:3295 writesame 0xbc113f1 thru 0xbc211f6 data_size 0x23(0xfe06 bytes) 2024-09-13T09:18:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:23.132 INFO:teuthology.orchestra.run.smithi032.stdout:3300 read 0x7871ed6 thru 0x7880b3c (0xec67 bytes) 2024-09-13T09:18:23.133 INFO:teuthology.orchestra.run.smithi032.stdout:3302 write 0xbbb19f1 thru 0xbbb66ab (0x4cbb bytes) 2024-09-13T09:18:23.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:18:23.140 INFO:teuthology.orchestra.run.smithi032.stdout:3303 punch from 0x56bd772 to 0x56c582e, (0x80bc bytes) 2024-09-13T09:18:23.154 INFO:teuthology.orchestra.run.smithi032.stdout:3304 write 0xc78ee0f thru 0xc792fcc (0x41be bytes) 2024-09-13T09:18:23.160 INFO:teuthology.orchestra.run.smithi032.stdout:3306 read 0x37b7e66 thru 0x37b8daf (0xf4a bytes) 2024-09-13T09:18:23.160 INFO:teuthology.orchestra.run.smithi032.stdout:3307 writesame 0x92852d7 thru 0x928c928 data_size 0x41(0x7652 bytes) 2024-09-13T09:18:23.163 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:23.163 INFO:teuthology.orchestra.run.smithi032.stdout:3309 read 0x62cdf6 thru 0x633dc6 (0x6fd1 bytes) 2024-09-13T09:18:23.164 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:23.193 INFO:teuthology.orchestra.run.smithi032.stdout:3312 trunc from 0xc792fcd to 0x1153961 2024-09-13T09:18:23.202 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:23.198+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:23.281 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:23.278+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:23.316 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:23.316 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:23.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:23.373 INFO:teuthology.orchestra.run.smithi032.stdout:3313 trunc from 0x1153961 to 0x867f23 2024-09-13T09:18:23.381 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:23.374+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:23.394 INFO:teuthology.orchestra.run.smithi032.stdout:3314 writesame 0x3349e7d thru 0x334c9a5 data_size 0x1d(0x2b29 bytes) 2024-09-13T09:18:23.399 INFO:teuthology.orchestra.run.smithi032.stdout:3315 trunc from 0x334c9a6 to 0x9e3461 2024-09-13T09:18:23.411 INFO:teuthology.orchestra.run.smithi032.stdout:3316 read 0x940594 thru 0x941c24 (0x1691 bytes) 2024-09-13T09:18:23.411 INFO:teuthology.orchestra.run.smithi032.stdout:3317 punch from 0x3dbd3f to 0x3e95e5, (0xd8a6 bytes) 2024-09-13T09:18:23.412 INFO:teuthology.orchestra.run.smithi032.stdout:3320 clone 22 order 21 su 1048576 sc 4 2024-09-13T09:18:23.466 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:23.466 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:23.480 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:23.527 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:23.526+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:23.583 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:23.582+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:23.644 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:23.644 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:23.658 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:23.703 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:23.702+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:23.784 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:23.778+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:23.831 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:23.831 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:23.845 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:23.891 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:23.890+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:23.970 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:23.971 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:23.985 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:24.027 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:24.026+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:24.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:24.119 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:24.118+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:18:24.170 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:24.170 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:24.186 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:24.189 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:24.182+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:24.227 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:24.227 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:24.245 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:24.245 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:24.247 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone21 from 0x9e3461 (overlap 0x867f23) to 0x5e25cb 2024-09-13T09:18:24.279 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:24.289 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:24.286+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:24.298 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:24.298 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:24.356 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:24.371 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-13T09:18:24.382 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:24.382 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:24.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:24.382 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:24.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.383 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:18:24.383 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:18:24.383 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.306952247000002, 2024-09-13T09:18:24.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:18:24.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.383 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:24.383 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.383 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.383 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.383 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:18:24.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.384 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:18:24.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:18:24.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:18:24.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:18:24.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:18:24.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:18:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:18:24.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:18:24.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.390 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.390 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29220 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1052672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.390 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.593818+0000", 2024-09-13T09:18:24.390 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7871384930000005, 2024-09-13T09:18:24.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032623085000000003, 2024-09-13T09:18:24.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.390 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29220 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593818+0000", 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593818+0000", 2024-09-13T09:18:24.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593817+0000", 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593823+0000", 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8370000000000002e-06 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593840+0000", 2024-09-13T09:18:24.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6875999999999999e-05 2024-09-13T09:18:24.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593854+0000", 2024-09-13T09:18:24.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4014e-05 2024-09-13T09:18:24.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626245+0000", 2024-09-13T09:18:24.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032391101999999998 2024-09-13T09:18:24.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626343+0000", 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7442000000000007e-05 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626441+0000", 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8863000000000005e-05 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29221 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.593865+0000", 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7870914449999997, 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032785591000000003, 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29221 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593865+0000", 2024-09-13T09:18:24.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593865+0000", 2024-09-13T09:18:24.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593864+0000", 2024-09-13T09:18:24.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:24.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593870+0000", 2024-09-13T09:18:24.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.694e-06 2024-09-13T09:18:24.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593885+0000", 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5065000000000001e-05 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593897+0000", 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1779e-05 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626478+0000", 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032581242000000003 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626578+0000", 2024-09-13T09:18:24.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9919000000000003e-05 2024-09-13T09:18:24.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626651+0000", 2024-09-13T09:18:24.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3182999999999994e-05 2024-09-13T09:18:24.399 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.399 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.399 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.400 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29222 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1110016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.400 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.593907+0000", 2024-09-13T09:18:24.400 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7870494419999998, 2024-09-13T09:18:24.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032975155999999999, 2024-09-13T09:18:24.400 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.400 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.400 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.400 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.400 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29222 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593907+0000", 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593907+0000", 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593906+0000", 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593912+0000", 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5910000000000003e-06 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593930+0000", 2024-09-13T09:18:24.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7918999999999999e-05 2024-09-13T09:18:24.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593942+0000", 2024-09-13T09:18:24.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1875999999999999e-05 2024-09-13T09:18:24.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626684+0000", 2024-09-13T09:18:24.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032742344 2024-09-13T09:18:24.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626781+0000", 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6879000000000005e-05 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626883+0000", 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101761 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29223 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1138688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.593953+0000", 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.78700423, 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033225364, 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29223 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593953+0000", 2024-09-13T09:18:24.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593953+0000", 2024-09-13T09:18:24.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593952+0000", 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593957+0000", 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593975+0000", 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7671000000000001e-05 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.593987+0000", 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2225000000000001e-05 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.626925+0000", 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032937578000000002 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627068+0000", 2024-09-13T09:18:24.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014354600000000001 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627178+0000", 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109622 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29224 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1167360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594001+0000", 2024-09-13T09:18:24.409 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7869556480000002, 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033434541999999998, 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29224 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594001+0000", 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594001+0000", 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594000+0000", 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594023+0000", 2024-09-13T09:18:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3030000000000001e-05 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594040+0000", 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7108e-05 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594053+0000", 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3185e-05 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627217+0000", 2024-09-13T09:18:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033163555999999997 2024-09-13T09:18:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627329+0000", 2024-09-13T09:18:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112562 2024-09-13T09:18:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627436+0000", 2024-09-13T09:18:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106538 2024-09-13T09:18:24.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29225 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1196032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594112+0000", 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7868450290000002, 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033646220999999997, 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29225 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594112+0000", 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594112+0000", 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594111+0000", 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594118+0000", 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1829999999999997e-06 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594136+0000", 2024-09-13T09:18:24.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8444000000000002e-05 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594164+0000", 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8238999999999998e-05 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627501+0000", 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033336688000000003 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627631+0000", 2024-09-13T09:18:24.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000130128 2024-09-13T09:18:24.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627758+0000", 2024-09-13T09:18:24.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012672200000000001 2024-09-13T09:18:24.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.418 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29226 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1224704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594176+0000", 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7867806210000001, 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033850475999999997, 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29226 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594176+0000", 2024-09-13T09:18:24.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594176+0000", 2024-09-13T09:18:24.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594175+0000", 2024-09-13T09:18:24.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594181+0000", 2024-09-13T09:18:24.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2389999999999999e-06 2024-09-13T09:18:24.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594197+0000", 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5464999999999998e-05 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594210+0000", 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3071000000000001e-05 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627828+0000", 2024-09-13T09:18:24.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033618289000000003 2024-09-13T09:18:24.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.627938+0000", 2024-09-13T09:18:24.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110136 2024-09-13T09:18:24.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628027+0000", 2024-09-13T09:18:24.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8418999999999995e-05 2024-09-13T09:18:24.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.423 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29227 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1253376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594221+0000", 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7867361669999999, 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034003004000000003, 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29227 2024-09-13T09:18:24.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.425 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594221+0000", 2024-09-13T09:18:24.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594221+0000", 2024-09-13T09:18:24.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594220+0000", 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594225+0000", 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5849999999999999e-06 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594241+0000", 2024-09-13T09:18:24.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5542000000000001e-05 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594255+0000", 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4396000000000001e-05 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628061+0000", 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033806151999999999 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628160+0000", 2024-09-13T09:18:24.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8400999999999995e-05 2024-09-13T09:18:24.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628224+0000", 2024-09-13T09:18:24.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3938999999999999e-05 2024-09-13T09:18:24.428 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.428 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.428 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.428 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29228 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.429 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594266+0000", 2024-09-13T09:18:24.429 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7866905470000001, 2024-09-13T09:18:24.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034238179000000001, 2024-09-13T09:18:24.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.429 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.429 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29228 2024-09-13T09:18:24.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.429 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594266+0000", 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594266+0000", 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594265+0000", 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594286+0000", 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1208000000000001e-05 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594308+0000", 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1563999999999999e-05 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594322+0000", 2024-09-13T09:18:24.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3742e-05 2024-09-13T09:18:24.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628275+0000", 2024-09-13T09:18:24.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033953637000000002 2024-09-13T09:18:24.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628393+0000", 2024-09-13T09:18:24.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117543 2024-09-13T09:18:24.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628504+0000", 2024-09-13T09:18:24.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111474 2024-09-13T09:18:24.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.433 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.433 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29229 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.433 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594333+0000", 2024-09-13T09:18:24.433 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7866235509999999, 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034422919000000003, 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29229 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594333+0000", 2024-09-13T09:18:24.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594333+0000", 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594332+0000", 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594338+0000", 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594354+0000", 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5727999999999999e-05 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594367+0000", 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2473000000000001e-05 2024-09-13T09:18:24.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628558+0000", 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034191322000000003 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628661+0000", 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103405 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628756+0000", 2024-09-13T09:18:24.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4943000000000002e-05 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29230 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1339392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594446+0000", 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7865103270000002, 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034555941, 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.438 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29230 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594446+0000", 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594446+0000", 2024-09-13T09:18:24.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594445+0000", 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594454+0000", 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3480000000000004e-06 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594474+0000", 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0438999999999999e-05 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594489+0000", 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4961e-05 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628813+0000", 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034323499 2024-09-13T09:18:24.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.628920+0000", 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107293 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629002+0000", 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2575000000000001e-05 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29231 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1368064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594501+0000", 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.786455353, 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034737715000000002, 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29231 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594501+0000", 2024-09-13T09:18:24.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594501+0000", 2024-09-13T09:18:24.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594500+0000", 2024-09-13T09:18:24.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:24.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594520+0000", 2024-09-13T09:18:24.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9542e-05 2024-09-13T09:18:24.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594537+0000", 2024-09-13T09:18:24.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7399000000000001e-05 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594551+0000", 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3653e-05 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629069+0000", 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034518189999999997 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629169+0000", 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100171 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629239+0000", 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0005999999999993e-05 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.447 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29232 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1396736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594563+0000", 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.78639422, 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986323, 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29232 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594563+0000", 2024-09-13T09:18:24.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594563+0000", 2024-09-13T09:18:24.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594561+0000", 2024-09-13T09:18:24.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594567+0000", 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594583+0000", 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6047000000000001e-05 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594596+0000", 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2619000000000001e-05 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629292+0000", 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034696236999999998 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629420+0000", 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012762599999999999 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629549+0000", 2024-09-13T09:18:24.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012909099999999999 2024-09-13T09:18:24.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.452 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.452 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29233 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.452 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594607+0000", 2024-09-13T09:18:24.452 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7863500339999998, 2024-09-13T09:18:24.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035129889999999997, 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29233 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594607+0000", 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594607+0000", 2024-09-13T09:18:24.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594606+0000", 2024-09-13T09:18:24.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594611+0000", 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594627+0000", 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5619000000000001e-05 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594639+0000", 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2398999999999999e-05 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629605+0000", 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034965969 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629671+0000", 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6099000000000005e-05 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629737+0000", 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5499000000000004e-05 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29234 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594650+0000", 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.786307077, 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035235401999999999, 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29234 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594650+0000", 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594650+0000", 2024-09-13T09:18:24.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594649+0000", 2024-09-13T09:18:24.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594654+0000", 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6280000000000002e-06 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594670+0000", 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6161999999999999e-05 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594697+0000", 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6284000000000001e-05 2024-09-13T09:18:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629769+0000", 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071946999999999 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629833+0000", 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4721999999999995e-05 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629885+0000", 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1749999999999997e-05 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29235 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1482752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594709+0000", 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7862474920000002, 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035355323000000001, 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.463 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.463 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29235 2024-09-13T09:18:24.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.463 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594709+0000", 2024-09-13T09:18:24.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594709+0000", 2024-09-13T09:18:24.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594708+0000", 2024-09-13T09:18:24.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594715+0000", 2024-09-13T09:18:24.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3049999999999998e-06 2024-09-13T09:18:24.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594732+0000", 2024-09-13T09:18:24.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7229999999999999e-05 2024-09-13T09:18:24.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594745+0000", 2024-09-13T09:18:24.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2694000000000001e-05 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629923+0000", 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035178216999999998 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.629997+0000", 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3971e-05 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630065+0000", 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7940000000000003e-05 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29236 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1511424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594756+0000", 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7862010179999999, 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035707192999999998, 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.467 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29236 2024-09-13T09:18:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594756+0000", 2024-09-13T09:18:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594756+0000", 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594755+0000", 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594760+0000", 2024-09-13T09:18:24.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6960000000000002e-06 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594777+0000", 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6239000000000002e-05 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594789+0000", 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2734e-05 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630196+0000", 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035406919000000002 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630337+0000", 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000140801 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630463+0000", 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012583200000000001 2024-09-13T09:18:24.471 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29237 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594815+0000", 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.7861422469999999, 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035904437999999997, 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.472 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29237 2024-09-13T09:18:24.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.473 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594815+0000", 2024-09-13T09:18:24.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594815+0000", 2024-09-13T09:18:24.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594813+0000", 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594820+0000", 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1480000000000003e-06 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594836+0000", 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6379e-05 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594848+0000", 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2231e-05 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630525+0000", 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035676280999999997 2024-09-13T09:18:24.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630631+0000", 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106099 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630719+0000", 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8362999999999994e-05 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.477 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:29238 3.4 3:219ba442:::rbd_data.1185833655de.0000000000000047:16 [read 1568768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.477 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.594914+0000", 2024-09-13T09:18:24.477 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.786042385, 2024-09-13T09:18:24.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036012464000000001, 2024-09-13T09:18:24.477 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 29238 2024-09-13T09:18:24.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.477 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594914+0000", 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594914+0000", 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594913+0000", 2024-09-13T09:18:24.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594920+0000", 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4429999999999997e-06 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594940+0000", 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9273e-05 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.594974+0000", 2024-09-13T09:18:24.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4403000000000002e-05 2024-09-13T09:18:24.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630755+0000", 2024-09-13T09:18:24.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035780660999999998 2024-09-13T09:18:24.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630854+0000", 2024-09-13T09:18:24.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9498000000000002e-05 2024-09-13T09:18:24.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.630927+0000", 2024-09-13T09:18:24.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2591999999999999e-05 2024-09-13T09:18:24.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.481 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.481 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.481 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:24.483 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:24.483 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:24.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:24.484 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:24.485 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:24.485 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:24.478+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:24.528 INFO:teuthology.orchestra.run.smithi032.stdout:3321 trunc from 0x9e3461 to 0x57396a1 2024-09-13T09:18:24.544 INFO:teuthology.orchestra.run.smithi032.stdout:3322 punch from 0x376dded to 0x3774091, (0x62a4 bytes) 2024-09-13T09:18:24.545 INFO:teuthology.orchestra.run.smithi032.stdout:3324 write 0x8e8a1c thru 0x8f4d13 (0xc2f8 bytes) 2024-09-13T09:18:24.572 INFO:teuthology.orchestra.run.smithi032.stdout:3326 write 0xbb52f4d thru 0xbb5ec43 (0xbcf7 bytes) 2024-09-13T09:18:24.577 INFO:teuthology.orchestra.run.smithi032.stdout:3328 read 0x24edac3 thru 0x24eddc0 (0x2fe bytes) 2024-09-13T09:18:24.577 INFO:teuthology.orchestra.run.smithi032.stdout:3329 write 0x8e9ca8b thru 0x8ea7342 (0xa8b8 bytes) 2024-09-13T09:18:24.583 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:24.582+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:24.586 INFO:teuthology.orchestra.run.smithi032.stdout:3330 trunc from 0xbb5ec44 to 0xe222da9 2024-09-13T09:18:24.611 INFO:teuthology.orchestra.run.smithi032.stdout:3331 trunc from 0xe222da9 to 0x4b9f8e8 2024-09-13T09:18:24.633 INFO:teuthology.orchestra.run.smithi032.stdout:3332 read 0x3cbf8a8 thru 0x3ccb98a (0xc0e3 bytes) 2024-09-13T09:18:24.633 INFO:teuthology.orchestra.run.smithi032.stdout:3333 read 0x316dcad thru 0x316f4cc (0x1820 bytes) 2024-09-13T09:18:24.633 INFO:teuthology.orchestra.run.smithi032.stdout:3335 punch from 0x2ab82e to 0x2ac6ba, (0xe8c bytes) 2024-09-13T09:18:24.657 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:24.657 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:24.657 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:24.657 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:24.658 INFO:teuthology.orchestra.run.smithi032.stdout:3336 write 0xe32a704 thru 0xe3380b5 (0xd9b2 bytes) 2024-09-13T09:18:24.666 INFO:teuthology.orchestra.run.smithi032.stdout:3337 writesame 0xc03e4f4 thru 0xc042891 data_size 0xf(0x439e bytes) 2024-09-13T09:18:24.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:24.674 INFO:teuthology.orchestra.run.smithi032.stdout:3342 write 0x93a79e7 thru 0x93b6cfd (0xf317 bytes) 2024-09-13T09:18:24.682 INFO:teuthology.orchestra.run.smithi032.stdout:3345 writesame 0x9bd9268 thru 0x9bda6a4 data_size 0x3(0x143d bytes) 2024-09-13T09:18:24.684 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:24.678+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:24.689 INFO:teuthology.orchestra.run.smithi032.stdout:3346 write 0x85c8069 thru 0x85d4a69 (0xca01 bytes) 2024-09-13T09:18:24.696 INFO:teuthology.orchestra.run.smithi032.stdout:3347 punch from 0xbd4aa4e to 0xbd4e7cd, (0x3d7f bytes) 2024-09-13T09:18:24.696 INFO:teuthology.orchestra.run.smithi032.stdout:3351 read 0xbaf06a4 thru 0xbafdaf1 (0xd44e bytes) 2024-09-13T09:18:24.696 INFO:teuthology.orchestra.run.smithi032.stdout:3352 write 0x9c8ff18 thru 0x9c96bdc (0x6cc5 bytes) 2024-09-13T09:18:24.702 INFO:teuthology.orchestra.run.smithi032.stdout:3353 writesame 0x255f496 thru 0x256479a data_size 0x1(0x5305 bytes) 2024-09-13T09:18:24.719 INFO:teuthology.orchestra.run.smithi032.stdout:3354 punch from 0xdfdf6ed to 0xdfe4358, (0x4c6b bytes) 2024-09-13T09:18:24.721 INFO:teuthology.orchestra.run.smithi032.stdout:3355 trunc from 0xe3380b6 to 0xcbec341 2024-09-13T09:18:24.760 INFO:teuthology.orchestra.run.smithi032.stdout:3356 trunc from 0xcbec341 to 0x365df82 2024-09-13T09:18:24.785 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:24.778+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:24.791 INFO:teuthology.orchestra.run.smithi032.stdout:3357 trunc from 0x365df82 to 0x8be5072 2024-09-13T09:18:24.803 INFO:teuthology.orchestra.run.smithi032.stdout:3360 write 0x4defd0e thru 0x4dfab1b (0xae0e bytes) 2024-09-13T09:18:24.810 INFO:teuthology.orchestra.run.smithi032.stdout:3361 read 0x22e5e53 thru 0x22ef144 (0x92f2 bytes) 2024-09-13T09:18:24.810 INFO:teuthology.orchestra.run.smithi032.stdout:3362 punch from 0x6d27870 to 0x6d332a4, (0xba34 bytes) 2024-09-13T09:18:24.815 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:24.816 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:24.819 INFO:teuthology.orchestra.run.smithi032.stdout:3363 trunc from 0x8be5072 to 0xcea71de 2024-09-13T09:18:24.830 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:24.830 INFO:teuthology.orchestra.run.smithi032.stdout:3366 punch from 0x2fb2ef to 0x30ad90, (0xfaa1 bytes) 2024-09-13T09:18:24.833 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:24.853 INFO:teuthology.orchestra.run.smithi032.stdout:3367 trunc from 0xcea71de to 0x2d1e720 2024-09-13T09:18:24.875 INFO:teuthology.orchestra.run.smithi032.stdout:3368 read 0x2cd3f39 thru 0x2cda5cd (0x6695 bytes) 2024-09-13T09:18:24.886 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:24.882+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:24.888 INFO:teuthology.orchestra.run.smithi032.stdout:3370 trunc from 0x2d1e720 to 0x86f6c6d 2024-09-13T09:18:24.898 INFO:teuthology.orchestra.run.smithi032.stdout:3371 write 0x222eb34 thru 0x223cb7b (0xe048 bytes) 2024-09-13T09:18:24.905 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:24.905 INFO:teuthology.orchestra.run.smithi032.stdout:3374 read 0x4445431 thru 0x44518ab (0xc47b bytes) 2024-09-13T09:18:24.905 INFO:teuthology.orchestra.run.smithi032.stdout:3376 writesame 0x6167a01 thru 0x616d126 data_size 0x61(0x5726 bytes) 2024-09-13T09:18:24.912 INFO:teuthology.orchestra.run.smithi032.stdout:3377 read 0x115767e thru 0x1163acc (0xc44f bytes) 2024-09-13T09:18:24.912 INFO:teuthology.orchestra.run.smithi032.stdout:3378 read 0x328471 thru 0x33631c (0xdeac bytes) 2024-09-13T09:18:24.914 INFO:teuthology.orchestra.run.smithi032.stdout:3379 read 0x4f0aec0 thru 0x4f1038e (0x54cf bytes) 2024-09-13T09:18:24.914 INFO:teuthology.orchestra.run.smithi032.stdout:3380 punch from 0x52026dc to 0x52027f8, (0x11c bytes) 2024-09-13T09:18:24.914 INFO:teuthology.orchestra.run.smithi032.stdout:3381 read 0x74ca8f7 thru 0x74cc142 (0x184c bytes) 2024-09-13T09:18:24.914 INFO:teuthology.orchestra.run.smithi032.stdout:3382 write 0x1e5c836 thru 0x1e61944 (0x510f bytes) 2024-09-13T09:18:24.931 INFO:teuthology.orchestra.run.smithi032.stdout:3383 trunc from 0x86f6c6d to 0x2b8def4 2024-09-13T09:18:24.978 INFO:teuthology.orchestra.run.smithi032.stdout:3387 write 0x8aeb35d thru 0x8af9419 (0xe0bd bytes) 2024-09-13T09:18:24.985 INFO:teuthology.orchestra.run.smithi032.stdout:3388 punch from 0x586fe48 to 0x587d9e6, (0xdb9e bytes) 2024-09-13T09:18:24.985 INFO:teuthology.orchestra.run.smithi032.stdout:3389 read 0x386deab thru 0x387986f (0xb9c5 bytes) 2024-09-13T09:18:24.985 INFO:teuthology.orchestra.run.smithi032.stdout:3391 read 0x55b7771 thru 0x55c172a (0x9fba bytes) 2024-09-13T09:18:24.986 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:24.987 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:24.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:24.987 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:24.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.987 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33548 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.987 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:18:24.987 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9941456769999997, 2024-09-13T09:18:24.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025760904000000001, 2024-09-13T09:18:24.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.987 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.988 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33548 2024-09-13T09:18:24.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.988 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:18:24.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990124+0000", 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990130+0000", 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990135+0000", 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007826+0000", 2024-09-13T09:18:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017691018999999999 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007896+0000", 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0059000000000001e-05 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.015883+0000", 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079875269999999995 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33750 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993520+0000", 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9907478589999998, 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023633346999999999, 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33750 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.992 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993520+0000", 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993520+0000", 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993520+0000", 2024-09-13T09:18:24.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993522+0000", 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.939e-06 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993528+0000", 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0480000000000004e-06 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993538+0000", 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8949999999999993e-06 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017046+0000", 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023508322000000002 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017108+0000", 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2250999999999997e-05 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017154+0000", 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5201000000000003e-05 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33751 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1593344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9907260229999997, 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023779101, 2024-09-13T09:18:24.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33751 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993548+0000", 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.776e-06 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993555+0000", 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993560+0000", 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017188+0000", 2024-09-13T09:18:24.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023627984000000001 2024-09-13T09:18:25.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017271+0000", 2024-09-13T09:18:25.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2798000000000002e-05 2024-09-13T09:18:25.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017321+0000", 2024-09-13T09:18:25.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0557999999999999e-05 2024-09-13T09:18:25.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.000 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33752 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.990703903, 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023925173000000001, 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.001 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33752 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993566+0000", 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993573+0000", 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.759e-06 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993578+0000", 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8339999999999998e-06 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017349+0000", 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023770959000000001 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017423+0000", 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3652000000000006e-05 2024-09-13T09:18:25.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017489+0000", 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6748999999999994e-05 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33753 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1650688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9906854210000002, 2024-09-13T09:18:25.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024066349000000001, 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33753 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993582+0000", 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993585+0000", 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:18:25.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993591+0000", 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993595+0000", 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017515+0000", 2024-09-13T09:18:25.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023920117000000001 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017583+0000", 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8485999999999994e-05 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017649+0000", 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33754 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9906691270000003, 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02420106, 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33754 2024-09-13T09:18:25.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993601+0000", 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993607+0000", 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9669999999999998e-06 2024-09-13T09:18:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993611+0000", 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017669+0000", 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024057683999999999 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017742+0000", 2024-09-13T09:18:25.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3397000000000002e-05 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017800+0000", 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7711999999999999e-05 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33755 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1708032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:18:25.014 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9906527079999998, 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024346923999999999, 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33755 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:18:25.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993617+0000", 2024-09-13T09:18:25.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:18:25.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993624+0000", 2024-09-13T09:18:25.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0469999999999999e-06 2024-09-13T09:18:25.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993628+0000", 2024-09-13T09:18:25.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017830+0000", 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024201879999999999 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017897+0000", 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7096999999999998e-05 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017962+0000", 2024-09-13T09:18:25.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5455000000000003e-05 2024-09-13T09:18:25.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.019 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.019 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33756 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.019 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9906359819999997, 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024485512000000001, 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33756 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993635+0000", 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993641+0000", 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993645+0000", 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2520000000000001e-06 2024-09-13T09:18:25.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017979+0000", 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024333950999999999 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018045+0000", 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5843e-05 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018118+0000", 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2417999999999997e-05 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33757 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1765376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9906185839999999, 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024605578999999999, 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.024 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33757 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:18:25.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993649+0000", 2024-09-13T09:18:25.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993651+0000", 2024-09-13T09:18:25.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.841e-06 2024-09-13T09:18:25.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993657+0000", 2024-09-13T09:18:25.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-13T09:18:25.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993662+0000", 2024-09-13T09:18:25.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:18:25.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018139+0000", 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024476748 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018196+0000", 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7188000000000002e-05 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018255+0000", 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9045999999999997e-05 2024-09-13T09:18:25.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33758 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9906019510000004, 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024736898, 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.029 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33758 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993668+0000", 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993674+0000", 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:18:25.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993679+0000", 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.515e-06 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018284+0000", 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024604816000000002 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018356+0000", 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2782999999999998e-05 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018403+0000", 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6726e-05 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33759 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1822720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:18:25.033 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9905853449999995, 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873635000000002, 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33759 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:18:25.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993682+0000", 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993685+0000", 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993691+0000", 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1840000000000004e-06 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993696+0000", 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018429+0000", 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024733330000000001 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018494+0000", 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5233000000000004e-05 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018556+0000", 2024-09-13T09:18:25.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2125999999999994e-05 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33760 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9905684900000002, 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025058980000000002, 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.038 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33760 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993699+0000", 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993702+0000", 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.632e-06 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993708+0000", 2024-09-13T09:18:25.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2280000000000003e-06 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993713+0000", 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018586+0000", 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873572 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018645+0000", 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9098000000000002e-05 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018759+0000", 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113224 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.042 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33761 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1880064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9905517929999998, 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025229076, 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33761 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:18:25.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993718+0000", 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.092e-06 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993724+0000", 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0839999999999997e-06 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993729+0000", 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:18:25.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018790+0000", 2024-09-13T09:18:25.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025061039 2024-09-13T09:18:25.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018871+0000", 2024-09-13T09:18:25.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0885999999999998e-05 2024-09-13T09:18:25.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018945+0000", 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4773999999999999e-05 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33762 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9905353190000001, 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025559609, 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.047 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.048 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.048 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.048 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.048 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33762 2024-09-13T09:18:25.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.048 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:18:25.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993732+0000", 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993735+0000", 2024-09-13T09:18:25.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993740+0000", 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993745+0000", 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5290000000000002e-06 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019043+0000", 2024-09-13T09:18:25.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025298355000000002 2024-09-13T09:18:25.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019134+0000", 2024-09-13T09:18:25.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0781999999999999e-05 2024-09-13T09:18:25.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019292+0000", 2024-09-13T09:18:25.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015829 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33763 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9905193780000001, 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025732657999999999, 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.052 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.054 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33763 2024-09-13T09:18:25.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.054 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:18:25.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993748+0000", 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993755+0000", 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:18:25.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993762+0000", 2024-09-13T09:18:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:18:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993767+0000", 2024-09-13T09:18:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7230000000000004e-06 2024-09-13T09:18:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019322+0000", 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025555564999999999 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019402+0000", 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9433999999999999e-05 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019481+0000", 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9739000000000003e-05 2024-09-13T09:18:25.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33764 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9904970039999998, 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025908526000000001, 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33764 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993773+0000", 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993779+0000", 2024-09-13T09:18:25.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9610000000000002e-06 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993783+0000", 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.459e-06 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019513+0000", 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025729399 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019609+0000", 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6504999999999998e-05 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019680+0000", 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0393000000000002e-05 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.062 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33765 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1994752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9904807690000004, 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026051009999999999, 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33765 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993789+0000", 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993795+0000", 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.088e-06 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993799+0000", 2024-09-13T09:18:25.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019704+0000", 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025904311999999999 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019766+0000", 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2829000000000004e-05 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019838+0000", 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1935999999999997e-05 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33766 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9904647139999998, 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026238971999999999, 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33766 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:18:25.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993805+0000", 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993811+0000", 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9000000000000003e-06 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993816+0000", 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.527e-06 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019877+0000", 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026061337 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019960+0000", 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3245999999999995e-05 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020042+0000", 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2053999999999997e-05 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.071 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33767 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.9904483040000001, 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026407013999999999, 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33767 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993819+0000", 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993822+0000", 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993828+0000", 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993832+0000", 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.493e-06 2024-09-13T09:18:25.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020072+0000", 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026239519999999999 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020151+0000", 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9278999999999998e-05 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020227+0000", 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5965999999999997e-05 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33768 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.990429851, 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026564424, 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33768 2024-09-13T09:18:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:18:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993840+0000", 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.996e-06 2024-09-13T09:18:25.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993846+0000", 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993858+0000", 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2214e-05 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020253+0000", 2024-09-13T09:18:25.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026394926999999999 2024-09-13T09:18:25.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020329+0000", 2024-09-13T09:18:25.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5193999999999999e-05 2024-09-13T09:18:25.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020403+0000", 2024-09-13T09:18:25.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4145000000000002e-05 2024-09-13T09:18:25.080 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.080 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.081 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.081 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:25.081 INFO:teuthology.orchestra.run.smithi032.stdout:3392 read 0x492ffe6 thru 0x4939865 (0x9880 bytes) 2024-09-13T09:18:25.081 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:25.081 INFO:teuthology.orchestra.run.smithi032.stdout:3394 trunc from 0x8af941a to 0x13d3f8d 2024-09-13T09:18:25.083 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:25.083 INFO:teuthology.orchestra.run.smithi032.stdout:3395 write 0x90b1530 thru 0x90b1966 (0x437 bytes) 2024-09-13T09:18:25.084 INFO:teuthology.orchestra.run.smithi032.stdout:3396 read 0x4cdfe8a thru 0x4ce0bb1 (0xd28 bytes) 2024-09-13T09:18:25.084 INFO:teuthology.orchestra.run.smithi032.stdout:3397 write 0x2c21e0f thru 0x2c2d847 (0xba39 bytes) 2024-09-13T09:18:25.084 INFO:teuthology.orchestra.run.smithi032.stdout:3398 read 0x1ab4a3 thru 0x1b4214 (0x8d72 bytes) 2024-09-13T09:18:25.084 INFO:teuthology.orchestra.run.smithi032.stdout:3399 punch from 0x1b5b878 to 0x1b68511, (0xcc99 bytes) 2024-09-13T09:18:25.084 INFO:teuthology.orchestra.run.smithi032.stdout:3400 read 0x69d6ebf thru 0x69d741d (0x55f bytes) 2024-09-13T09:18:25.084 INFO:teuthology.orchestra.run.smithi032.stdout:3401 writesame 0x7fba820 thru 0x7fbb81b data_size 0xc(0xffc bytes) 2024-09-13T09:18:25.084 INFO:teuthology.orchestra.run.smithi032.stdout:3402 writesame 0x4971dac thru 0x4980fce data_size 0x1(0xf223 bytes) 2024-09-13T09:18:25.084 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:24.982+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:25.127 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:25.126+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:25.160 INFO:teuthology.orchestra.run.smithi032.stdout:3403 read 0x44f354d thru 0x44f6c14 (0x36c8 bytes) 2024-09-13T09:18:25.160 INFO:teuthology.orchestra.run.smithi032.stdout:3404 write 0x88ba6ab thru 0x88bd835 (0x318b bytes) 2024-09-13T09:18:25.167 INFO:teuthology.orchestra.run.smithi032.stdout:3405 read 0x808028f thru 0x808a05c (0x9dce bytes) 2024-09-13T09:18:25.178 INFO:teuthology.orchestra.run.smithi032.stdout:3406 trunc from 0x90b1967 to 0x3803e65 2024-09-13T09:18:25.188 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:25.182+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:25.198 INFO:teuthology.orchestra.run.smithi032.stdout:3407 read 0x368fb2c thru 0x369887f (0x8d54 bytes) 2024-09-13T09:18:25.198 INFO:teuthology.orchestra.run.smithi032.stdout:3411 read 0x4443ea thru 0x4528c1 (0xe4d8 bytes) 2024-09-13T09:18:25.210 INFO:teuthology.orchestra.run.smithi032.stdout:3412 trunc from 0x3803e65 to 0xa738984 2024-09-13T09:18:25.220 INFO:teuthology.orchestra.run.smithi032.stdout:3413 write 0x8f25f85 thru 0x8f29518 (0x3594 bytes) 2024-09-13T09:18:25.229 INFO:teuthology.orchestra.run.smithi032.stdout:3415 read 0x1ca53a5 thru 0x1ca53c7 (0x23 bytes) 2024-09-13T09:18:25.229 INFO:teuthology.orchestra.run.smithi032.stdout:3416 write 0x633c735 thru 0x6340be9 (0x44b5 bytes) 2024-09-13T09:18:25.235 INFO:teuthology.orchestra.run.smithi032.stdout:3418 read 0x1e7b7a8 thru 0x1e7e16d (0x29c6 bytes) 2024-09-13T09:18:25.250 INFO:teuthology.orchestra.run.smithi032.stdout:3419 writesame 0xca0f28f thru 0xca19b05 data_size 0x3d(0xa877 bytes) 2024-09-13T09:18:25.250 INFO:teuthology.orchestra.run.smithi032.stdout:3420 writesame 0xbbf0d3b thru 0xbbf3898 data_size 0x1a(0x2b5e bytes) 2024-09-13T09:18:25.256 INFO:teuthology.orchestra.run.smithi032.stdout:3421 read 0xbdfac5a thru 0xbe01e5e (0x7205 bytes) 2024-09-13T09:18:25.256 INFO:teuthology.orchestra.run.smithi032.stdout:3425 read 0x70aa494 thru 0x70b76e8 (0xd255 bytes) 2024-09-13T09:18:25.256 INFO:teuthology.orchestra.run.smithi032.stdout:3427 read 0x7ad4f95 thru 0x7ad75fb (0x2667 bytes) 2024-09-13T09:18:25.257 INFO:teuthology.orchestra.run.smithi032.stdout:3429 read 0x4c570b0 thru 0x4c65a4c (0xe99d bytes) 2024-09-13T09:18:25.257 INFO:teuthology.orchestra.run.smithi032.stdout:3430 writesame 0x4a4805c thru 0x4a51f44 data_size 0x1(0x9ee9 bytes) 2024-09-13T09:18:25.268 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:49433 3.b 3:d11175df:::rbd_data.1185e728d948.0000000000000026:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-13T09:18:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:25.261380+0000", 2024-09-13T09:18:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.007657077, 2024-09-13T09:18:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0076801329999999996, 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 49433 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:25.261380+0000", 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:25.261380+0000", 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:25.261377+0000", 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:25.261391+0000", 2024-09-13T09:18:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3573e-05 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:25.261422+0000", 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1344999999999999e-05 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:25.261436+0000", 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4100000000000001e-05 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:25.261552+0000", 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011581799999999999 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:25.261771+0000", 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021917200000000001 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:18:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 1 2024-09-13T09:18:25.273 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:25.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:25.289 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:25.282+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:25.302 INFO:teuthology.orchestra.run.smithi032.stdout:3433 read 0x6234678 thru 0x6241355 (0xccde bytes) 2024-09-13T09:18:25.303 INFO:teuthology.orchestra.run.smithi032.stdout:3434 write 0xb939a93 thru 0xb94678b (0xccf9 bytes) 2024-09-13T09:18:25.311 INFO:teuthology.orchestra.run.smithi032.stdout:3435 write 0x877c10f thru 0x878a4de (0xe3d0 bytes) 2024-09-13T09:18:25.319 INFO:teuthology.orchestra.run.smithi032.stdout:3436 read 0x91f50bb thru 0x91fc1d9 (0x711f bytes) 2024-09-13T09:18:25.336 INFO:teuthology.orchestra.run.smithi032.stdout:3437 trunc from 0xca19b06 to 0x1658adc 2024-09-13T09:18:25.355 INFO:teuthology.orchestra.run.smithi032.stdout:3439 read 0x1036f62 thru 0x1044dfd (0xde9c bytes) 2024-09-13T09:18:25.365 INFO:teuthology.orchestra.run.smithi032.stdout:3440 write 0x27ebade thru 0x27f938c (0xd8af bytes) 2024-09-13T09:18:25.390 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:25.386+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:25.400 INFO:teuthology.orchestra.run.smithi032.stdout:3441 write 0xc7c45c7 thru 0xc7c9288 (0x4cc2 bytes) 2024-09-13T09:18:25.407 INFO:teuthology.orchestra.run.smithi032.stdout:3442 punch from 0x57c1a8d to 0x57d0bbd, (0xf130 bytes) 2024-09-13T09:18:25.407 INFO:teuthology.orchestra.run.smithi032.stdout:3444 read 0xbe93706 thru 0xbe96954 (0x324f bytes) 2024-09-13T09:18:25.407 INFO:teuthology.orchestra.run.smithi032.stdout:3445 read 0xab5533a thru 0xab5958a (0x4251 bytes) 2024-09-13T09:18:25.408 INFO:teuthology.orchestra.run.smithi032.stdout:3447 flatten 2024-09-13T09:18:25.454 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:25.454 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:25.454 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:25.455 INFO:teuthology.orchestra.run.smithi032.stdout:3448 write 0x14f4531 thru 0x14f489b (0x36b bytes) 2024-09-13T09:18:25.463 INFO:teuthology.orchestra.run.smithi032.stdout:3449 write 0x4b16aaf thru 0x4b18f7d (0x24cf bytes) 2024-09-13T09:18:25.469 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:25.471 INFO:teuthology.orchestra.run.smithi032.stdout:3450 write 0x1daacae thru 0x1db88e5 (0xdc38 bytes) 2024-09-13T09:18:25.478 INFO:teuthology.orchestra.run.smithi032.stdout:3452 read 0x8ad4934 thru 0x8ad8660 (0x3d2d bytes) 2024-09-13T09:18:25.478 INFO:teuthology.orchestra.run.smithi032.stdout:3454 punch from 0xb7340ec to 0xb73d039, (0x8f4d bytes) 2024-09-13T09:18:25.478 INFO:teuthology.orchestra.run.smithi032.stdout:3455 clone 23 order 25 su 1048576 sc 9 2024-09-13T09:18:25.491 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:25.486+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:25.592 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:25.590+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:25.599 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:25.599 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:25.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:25.599 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:25.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.599 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:25.600 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:25.600 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.06097501, 2024-09-13T09:18:25.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:18:25.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.600 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:25.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.600 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:18:25.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.600 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:18:25.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:25.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:18:25.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:18:25.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:18:25.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:18:25.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:18:25.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:18:25.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:25.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:18:25.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:18:25.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:25.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:18:25.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:18:25.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:18:25.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.606 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32899 3.18 3:1fefe8db:::rbd_data.1185833655de.000000000000001e:16 [read 704512~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.606 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:25.606 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6914279620000006, 2024-09-13T09:18:25.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046124828, 2024-09-13T09:18:25.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32899 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905907+0000", 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905909+0000", 2024-09-13T09:18:25.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:18:25.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905919+0000", 2024-09-13T09:18:25.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0520000000000007e-06 2024-09-13T09:18:25.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905923+0000", 2024-09-13T09:18:25.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:18:25.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.951907+0000", 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.045983221999999997 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.951969+0000", 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2442999999999998e-05 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952032+0000", 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999998e-05 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6914080249999994, 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.611 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:25.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:18:25.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:18:25.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:18:25.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:18:25.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:18:25.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:18:25.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:18:25.615 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.615 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.615 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.615 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6913907039999998, 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:18:25.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6913737629999996, 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.623 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.624 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.624 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:25.624 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6913565350000006, 2024-09-13T09:18:25.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:18:25.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.624 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.624 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.624 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.624 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.625 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:18:25.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.625 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:25.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:18:25.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:18:25.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:18:25.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:18:25.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:18:25.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:18:25.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:18:25.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:18:25.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.628 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.628 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6913201139999998, 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:25.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:25.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:25.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:18:25.632 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6912666519999995, 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.633 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:18:25.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:18:25.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:18:25.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:18:25.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:18:25.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:18:25.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:18:25.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:18:25.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:18:25.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:18:25.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:18:25.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.637 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6912446320000001, 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:18:25.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:18:25.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:18:25.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:18:25.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:18:25.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6912199969999993, 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:18:25.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:18:25.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:18:25.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.646 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.646 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.646 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:25.646 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6912014739999996, 2024-09-13T09:18:25.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:18:25.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.646 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:18:25.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6911836030000007, 2024-09-13T09:18:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:25.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:25.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:18:25.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:18:25.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:18:25.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:18:25.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:18:25.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6911662730000003, 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:18:25.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:18:25.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:18:25.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:18:25.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:18:25.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:18:25.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6911488030000008, 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:18:25.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:25.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:18:25.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.663 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6911319369999998, 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:25.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:25.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:18:25.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:18:25.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:18:25.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.668 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.668 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.668 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:25.668 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6911141500000006, 2024-09-13T09:18:25.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:18:25.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.668 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:18:25.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.6910698400000008, 2024-09-13T09:18:25.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:18:25.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:18:25.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:18:25.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:18:25.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.676 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4579393520000004, 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.677 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:25.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:18:25.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:18:25.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:18:25.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:18:25.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:18:25.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:18:25.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:18:25.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4579225190000002, 2024-09-13T09:18:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:18:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:18:25.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:25.686 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.4579061610000004, 2024-09-13T09:18:25.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:18:25.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:25.687 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:25.687 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:25.687 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:25.687 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:25.687 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:18:25.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.687 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:25.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:25.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:18:25.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:25.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:18:25.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:18:25.691 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.691 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.691 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.691 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:25.691 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:25.691 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:25.692 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:25.735 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:25.734+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:25.795 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:25.790+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:25.826 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:25.826 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:25.826 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:25.826 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:25.840 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:25.934 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:25.930+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:25.997 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:25.990+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:26.013 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:26.014 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:26.014 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:26.014 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:26.014 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:26.029 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:26.098 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:26.094+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:26.151 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:18:26.197 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:18:26.197 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:18:26.197 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-13T09:18:26.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-09-13T09:18:26.199 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:26.194+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:26.207 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4388 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.218029825999999, 2024-09-13T09:18:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069349509000000004, 2024-09-13T09:18:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4388 2024-09-13T09:18:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985769+0000", 2024-09-13T09:18:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985775+0000", 2024-09-13T09:18:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5230000000000001e-06 2024-09-13T09:18:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985794+0000", 2024-09-13T09:18:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8425999999999999e-05 2024-09-13T09:18:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985809+0000", 2024-09-13T09:18:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.486e-05 2024-09-13T09:18:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.054965+0000", 2024-09-13T09:18:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069156463000000001 2024-09-13T09:18:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055036+0000", 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0780999999999999e-05 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055120+0000", 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3486000000000007e-05 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4389 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217979303, 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069473833999999998, 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4389 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985820+0000", 2024-09-13T09:18:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985859+0000", 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9279999999999997e-05 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985881+0000", 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2116999999999998e-05 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985896+0000", 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4698999999999999e-05 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055152+0000", 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069256568000000004 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055220+0000", 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8255000000000003e-05 2024-09-13T09:18:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055294+0000", 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3858999999999999e-05 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4390 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217883690999997, 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069563635999999998, 2024-09-13T09:18:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4390 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985915+0000", 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985922+0000", 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:18:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985941+0000", 2024-09-13T09:18:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8260000000000001e-05 2024-09-13T09:18:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985955+0000", 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4409999999999999e-05 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055325+0000", 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069370290000000001 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055395+0000", 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0158000000000002e-05 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055480+0000", 2024-09-13T09:18:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4340999999999998e-05 2024-09-13T09:18:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217832624000003, 2024-09-13T09:18:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:18:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:18:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:18:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:18:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:18:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:18:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:18:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:18:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:18:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217783888, 2024-09-13T09:18:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:18:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:18:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:18:26.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:18:26.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:18:26.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:18:26.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:18:26.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:18:26.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:18:26.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:18:26.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:18:26.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:18:26.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:18:26.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:18:26.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:18:26.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.233 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.233 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.233 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:26.233 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217716219000003, 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:26.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:18:26.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:18:26.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:18:26.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:18:26.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:18:26.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:18:26.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:18:26.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.238 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.238 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.238 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:26.238 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217576010000002, 2024-09-13T09:18:26.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:18:26.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.239 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:18:26.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:18:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:18:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:18:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:18:26.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:18:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:18:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:18:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:18:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.244 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.244 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:26.244 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217485144000001, 2024-09-13T09:18:26.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:18:26.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:26.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:18:26.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:18:26.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:18:26.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:18:26.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:18:26.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:18:26.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:18:26.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:18:26.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:18:26.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:18:26.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.249 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.249 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.249 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:26.249 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217410284000003, 2024-09-13T09:18:26.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:18:26.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.250 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.250 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:18:26.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.250 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:18:26.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:18:26.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:18:26.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:18:26.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:18:26.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:18:26.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:18:26.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:18:26.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:18:26.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:18:26.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:18:26.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:18:26.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.254 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.254 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217339107999997, 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:26.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:26.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:18:26.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:18:26.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:18:26.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:18:26.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:18:26.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.259 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:26.260 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217290054999999, 2024-09-13T09:18:26.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:18:26.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.260 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:18:26.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.260 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:26.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:26.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:18:26.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:18:26.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:18:26.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:18:26.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:18:26.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:18:26.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:18:26.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:18:26.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:18:26.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:18:26.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:26.267 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217228319, 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:26.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:18:26.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:18:26.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:18:26.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:18:26.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:18:26.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:18:26.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:18:26.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:18:26.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.272 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.272 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.272 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:26.272 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217178648999997, 2024-09-13T09:18:26.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:18:26.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.272 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:18:26.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:18:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:18:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:18:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:18:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217130130000001, 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:18:26.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:18:26.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:18:26.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:18:26.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:18:26.281 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.281 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.281 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.281 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.281 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217081733999997, 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:18:26.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:18:26.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:18:26.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:18:26.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:18:26.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.217034447000003, 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.286 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:18:26.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.291 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.291 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:26.291 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.216951045000002, 2024-09-13T09:18:26.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:18:26.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.291 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.291 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.291 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.291 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:18:26.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:18:26.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:18:26.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:18:26.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:18:26.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:18:26.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:18:26.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:18:26.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:18:26.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:18:26.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.295 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.216877318999998, 2024-09-13T09:18:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:18:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:18:26.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.297 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:26.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:26.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:18:26.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:18:26.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:18:26.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.300 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.216809196, 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:18:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.302 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:26.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:26.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:18:26.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.305 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.216758728999999, 2024-09-13T09:18:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:18:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:18:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:26.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:26.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:26.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:26.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:26.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:18:26.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:18:26.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.310 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:26.310 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:26.310 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:26.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:26.312 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:26.310+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:26.413 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:26.406+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:26.419 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-13T09:18:26.437 INFO:teuthology.orchestra.run.smithi032.stdout:3456 punch from 0x6864b33 to 0x68651fd, (0x6ca bytes) 2024-09-13T09:18:26.446 INFO:teuthology.orchestra.run.smithi032.stdout:3457 trunc from 0xc7c9289 to 0xea5fcae 2024-09-13T09:18:26.455 INFO:teuthology.orchestra.run.smithi032.stdout:3460 read 0x2b59ad2 thru 0x2b6938c (0xf8bb bytes) 2024-09-13T09:18:26.455 INFO:teuthology.orchestra.run.smithi032.stdout:3463 read 0xde04b56 thru 0xde14324 (0xf7cf bytes) 2024-09-13T09:18:26.455 INFO:teuthology.orchestra.run.smithi032.stdout:3464 writesame 0xb0460de thru 0xb0538fc data_size 0x1(0xd81f bytes) 2024-09-13T09:18:26.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:18:26.490 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:26.490 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:26.490 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:26.490 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:26.507 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:26.526 INFO:teuthology.orchestra.run.smithi032.stdout:3465 read 0xa451046 thru 0xa45e236 (0xd1f1 bytes) 2024-09-13T09:18:26.533 INFO:teuthology.orchestra.run.smithi032.stdout:3466 trunc from 0xea5fcae to 0xa19f9ba 2024-09-13T09:18:26.535 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-09-13T09:18:26.534+0000 7f1d86e9f640 -1 mon.a@0(leader).osd e71 definitely_dead 0 2024-09-13T09:18:26.550 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:26.546+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:26.559 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:26.559 INFO:teuthology.orchestra.run.smithi032.stdout:3469 punch from 0xbca4ef to 0xbd576e, (0xb27f bytes) 2024-09-13T09:18:26.559 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:26.559 INFO:teuthology.orchestra.run.smithi032.stdout:3471 write 0x351a93c thru 0x35281cb (0xd890 bytes) 2024-09-13T09:18:26.561 INFO:teuthology.orchestra.run.smithi032.stdout:3472 write 0x6405b4a thru 0x6409c10 (0x40c7 bytes) 2024-09-13T09:18:26.563 INFO:teuthology.orchestra.run.smithi032.stdout:3473 punch from 0x70f96af to 0x70fcc21, (0x3572 bytes) 2024-09-13T09:18:26.575 INFO:teuthology.orchestra.run.smithi032.stdout:3474 writesame 0xca6a5c4 thru 0xca6a5e4 data_size 0x1(0x21 bytes) 2024-09-13T09:18:26.580 INFO:teuthology.orchestra.run.smithi032.stdout:3475 trunc from 0xca6a5e5 to 0xe805348 2024-09-13T09:18:26.591 INFO:teuthology.orchestra.run.smithi032.stdout:3476 trunc from 0xe805348 to 0xbb3a160 2024-09-13T09:18:26.600 INFO:teuthology.orchestra.run.smithi032.stdout:3478 write 0x761fd82 thru 0x7628385 (0x8604 bytes) 2024-09-13T09:18:26.612 INFO:teuthology.orchestra.run.smithi032.stdout:3479 write 0xd575acb thru 0xd57a7b7 (0x4ced bytes) 2024-09-13T09:18:26.614 INFO:teuthology.orchestra.run.smithi032.stdout:3481 clone 24 order 22 su 1048576 sc 14 2024-09-13T09:18:26.614 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:26.610+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:26.653 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:26.653 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:26.669 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:26.670 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:26.670 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:26.670 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:26.684 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:26.687 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:26.704 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:26.715 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:26.714+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:26.722 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:26.740 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:26.740 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:26.759 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:26.759 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:26.779 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:26.779 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:26.805 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:26.805 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:26.817 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:26.810+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:26.820 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:26.918 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:26.914+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:26.989 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:26.990 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:27.004 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:27.046 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:27.042+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:27.119 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:27.114+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:18:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:marked in osd.6. 2024-09-13T09:18:27.176 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-13T09:18:27.179 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:27.179 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:27.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:27.220 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:27.218+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:27.322 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:27.318+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:27.365 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:27.365 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:27.379 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:27.423 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:27.422+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:27.505 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:27.506 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:27.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:27.565 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:27.562+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:27.625 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:27.622+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:27.639 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:27.639 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:27.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:27.727 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:27.726+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:27.827 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:27.822+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:27.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:27.929 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:27.926+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:27.975 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:27.975 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:27.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:28.030 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:28.026+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:28.131 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:28.126+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:28.174 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:28.201 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:28.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:28.247 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:28.246+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:28.323 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:28.323 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:28.334 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:28.330+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:28.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:28.435 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:28.434+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:28.506 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:28.506 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:28.520 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:28.536 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:28.534+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:28.637 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:28.630+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:28.690 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:28.690 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:28.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:28.738 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:28.734+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:28.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:28.832 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:28.832 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:28.839 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:28.834+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:28.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:28.939 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:28.934+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:28.943 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:28.943 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:28.960 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:28.960 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:28.976 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:28.977 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:28.992 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:28.993 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:29.012 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:29.013 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:29.015 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:29.015 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:29.015 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:29.015 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:29.029 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:29.033 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:29.033 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:29.039 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:29.034+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:29.054 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:29.075 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:29.140 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:29.134+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:29.154 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:29.154 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:29.155 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:29.155 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:29.155 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:29.169 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:29.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:18:29.263 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:29.262+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:29.296 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33548 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:18:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.305177148, 2024-09-13T09:18:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025760904000000001, 2024-09-13T09:18:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33548 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:18:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:18:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990124+0000", 2024-09-13T09:18:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:18:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990130+0000", 2024-09-13T09:18:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:18:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990135+0000", 2024-09-13T09:18:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:18:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007826+0000", 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017691018999999999 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007896+0000", 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0059000000000001e-05 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.015883+0000", 2024-09-13T09:18:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079875269999999995 2024-09-13T09:18:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33750 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993520+0000", 2024-09-13T09:18:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.30177933, 2024-09-13T09:18:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023633346999999999, 2024-09-13T09:18:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33750 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993520+0000", 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993520+0000", 2024-09-13T09:18:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993520+0000", 2024-09-13T09:18:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993522+0000", 2024-09-13T09:18:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.939e-06 2024-09-13T09:18:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993528+0000", 2024-09-13T09:18:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0480000000000004e-06 2024-09-13T09:18:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993538+0000", 2024-09-13T09:18:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8949999999999993e-06 2024-09-13T09:18:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017046+0000", 2024-09-13T09:18:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023508322000000002 2024-09-13T09:18:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017108+0000", 2024-09-13T09:18:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2250999999999997e-05 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017154+0000", 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5201000000000003e-05 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33751 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1593344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:18:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301757494, 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023779101, 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33751 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:18:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993548+0000", 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.776e-06 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993555+0000", 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993560+0000", 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:18:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017188+0000", 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023627984000000001 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017271+0000", 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2798000000000002e-05 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017321+0000", 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0557999999999999e-05 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33752 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301735374, 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023925173000000001, 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33752 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:18:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:18:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993566+0000", 2024-09-13T09:18:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:18:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993573+0000", 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.759e-06 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993578+0000", 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8339999999999998e-06 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017349+0000", 2024-09-13T09:18:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023770959000000001 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017423+0000", 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3652000000000006e-05 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017489+0000", 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6748999999999994e-05 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33753 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1650688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:18:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301716892, 2024-09-13T09:18:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024066349000000001, 2024-09-13T09:18:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33753 2024-09-13T09:18:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:18:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:18:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993582+0000", 2024-09-13T09:18:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993585+0000", 2024-09-13T09:18:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993591+0000", 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993595+0000", 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017515+0000", 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023920117000000001 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017583+0000", 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8485999999999994e-05 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017649+0000", 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33754 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:18:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301700598, 2024-09-13T09:18:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02420106, 2024-09-13T09:18:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33754 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:18:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:18:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993601+0000", 2024-09-13T09:18:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:18:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993607+0000", 2024-09-13T09:18:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9669999999999998e-06 2024-09-13T09:18:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993611+0000", 2024-09-13T09:18:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:18:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017669+0000", 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024057683999999999 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017742+0000", 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3397000000000002e-05 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017800+0000", 2024-09-13T09:18:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7711999999999999e-05 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33755 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1708032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301684179, 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024346923999999999, 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33755 2024-09-13T09:18:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:18:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993617+0000", 2024-09-13T09:18:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993624+0000", 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0469999999999999e-06 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993628+0000", 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017830+0000", 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024201879999999999 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017897+0000", 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7096999999999998e-05 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017962+0000", 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5455000000000003e-05 2024-09-13T09:18:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33756 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301667453, 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024485512000000001, 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33756 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993635+0000", 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993641+0000", 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:18:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993645+0000", 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2520000000000001e-06 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017979+0000", 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024333950999999999 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018045+0000", 2024-09-13T09:18:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5843e-05 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018118+0000", 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2417999999999997e-05 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33757 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1765376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:18:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301650055, 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024605578999999999, 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33757 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:18:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993649+0000", 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993651+0000", 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.841e-06 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993657+0000", 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993662+0000", 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:18:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018139+0000", 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024476748 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018196+0000", 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7188000000000002e-05 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018255+0000", 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9045999999999997e-05 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33758 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301633422, 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024736898, 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33758 2024-09-13T09:18:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:18:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:18:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:18:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993668+0000", 2024-09-13T09:18:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:18:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993674+0000", 2024-09-13T09:18:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:18:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993679+0000", 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.515e-06 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018284+0000", 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024604816000000002 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018356+0000", 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2782999999999998e-05 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018403+0000", 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6726e-05 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33759 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1822720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301616815999999, 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873635000000002, 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33759 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993682+0000", 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993685+0000", 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993691+0000", 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1840000000000004e-06 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993696+0000", 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018429+0000", 2024-09-13T09:18:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024733330000000001 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018494+0000", 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5233000000000004e-05 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018556+0000", 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2125999999999994e-05 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33760 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301599961000001, 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025058980000000002, 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33760 2024-09-13T09:18:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:18:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:18:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993699+0000", 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993702+0000", 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.632e-06 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993708+0000", 2024-09-13T09:18:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2280000000000003e-06 2024-09-13T09:18:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993713+0000", 2024-09-13T09:18:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:18:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018586+0000", 2024-09-13T09:18:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873572 2024-09-13T09:18:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018645+0000", 2024-09-13T09:18:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9098000000000002e-05 2024-09-13T09:18:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018759+0000", 2024-09-13T09:18:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113224 2024-09-13T09:18:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33761 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1880064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301583264, 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025229076, 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33761 2024-09-13T09:18:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:18:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:18:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993718+0000", 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.092e-06 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993724+0000", 2024-09-13T09:18:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0839999999999997e-06 2024-09-13T09:18:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993729+0000", 2024-09-13T09:18:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:18:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018790+0000", 2024-09-13T09:18:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025061039 2024-09-13T09:18:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018871+0000", 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0885999999999998e-05 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018945+0000", 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4773999999999999e-05 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33762 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301566790000001, 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025559609, 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33762 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993732+0000", 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993735+0000", 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993740+0000", 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993745+0000", 2024-09-13T09:18:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5290000000000002e-06 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019043+0000", 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025298355000000002 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019134+0000", 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0781999999999999e-05 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019292+0000", 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015829 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33763 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301550849, 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025732657999999999, 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33763 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993748+0000", 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993755+0000", 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993762+0000", 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993767+0000", 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7230000000000004e-06 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019322+0000", 2024-09-13T09:18:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025555564999999999 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019402+0000", 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9433999999999999e-05 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019481+0000", 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9739000000000003e-05 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33764 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301528475, 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025908526000000001, 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33764 2024-09-13T09:18:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993773+0000", 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993779+0000", 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9610000000000002e-06 2024-09-13T09:18:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993783+0000", 2024-09-13T09:18:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.459e-06 2024-09-13T09:18:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019513+0000", 2024-09-13T09:18:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025729399 2024-09-13T09:18:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019609+0000", 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6504999999999998e-05 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019680+0000", 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0393000000000002e-05 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33765 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1994752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301512239999999, 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026051009999999999, 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33765 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993789+0000", 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993795+0000", 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.088e-06 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993799+0000", 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019704+0000", 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025904311999999999 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019766+0000", 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2829000000000004e-05 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019838+0000", 2024-09-13T09:18:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1935999999999997e-05 2024-09-13T09:18:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33766 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:18:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301496185, 2024-09-13T09:18:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026238971999999999, 2024-09-13T09:18:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33766 2024-09-13T09:18:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:18:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993805+0000", 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:18:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993811+0000", 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9000000000000003e-06 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993816+0000", 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.527e-06 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019877+0000", 2024-09-13T09:18:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026061337 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019960+0000", 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3245999999999995e-05 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020042+0000", 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2053999999999997e-05 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33767 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301479775000001, 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026407013999999999, 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33767 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993819+0000", 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993822+0000", 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993828+0000", 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993832+0000", 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.493e-06 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020072+0000", 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026239519999999999 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020151+0000", 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9278999999999998e-05 2024-09-13T09:18:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020227+0000", 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5965999999999997e-05 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33768 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.301461322, 2024-09-13T09:18:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026564424, 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33768 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993840+0000", 2024-09-13T09:18:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.996e-06 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993846+0000", 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993858+0000", 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2214e-05 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020253+0000", 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026394926999999999 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020329+0000", 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5193999999999999e-05 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020403+0000", 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4145000000000002e-05 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.393 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.393 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:29.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:29.395 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:29.390+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:29.495 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:29.490+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:29.558 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:29.558 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:29.558 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:29.558 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:29.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:29.595 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:29.594+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:29.694 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:29.694 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:29.694 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:29.694 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:29.694 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:29.697 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:29.690+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:29.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:29.797 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:29.794+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:29.801 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone23 from 0xd57a7b8 (overlap 0xa19f9ba) to 0x40a63e4 2024-09-13T09:18:29.853 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:29.853 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:29.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:29.853 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:29.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.854 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:29.854 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:29.854 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.313680048999998, 2024-09-13T09:18:29.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:18:29.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.855 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:29.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.856 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:18:29.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.857 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:29.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:29.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:18:29.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:29.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:18:29.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:18:29.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:18:29.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:18:29.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:18:29.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:18:29.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:18:29.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:18:29.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:18:29.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:18:29.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:29.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:18:29.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:18:29.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:29.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:18:29.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:18:29.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:29.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:18:29.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:18:29.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:29.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:18:29.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:18:29.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:18:29.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:18:29.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.862 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.863 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32899 3.18 3:1fefe8db:::rbd_data.1185833655de.000000000000001e:16 [read 704512~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.863 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:29.863 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.944133001000001, 2024-09-13T09:18:29.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046124828, 2024-09-13T09:18:29.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.863 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.863 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.863 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.863 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.863 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32899 2024-09-13T09:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905907+0000", 2024-09-13T09:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905909+0000", 2024-09-13T09:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905919+0000", 2024-09-13T09:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0520000000000007e-06 2024-09-13T09:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905923+0000", 2024-09-13T09:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.951907+0000", 2024-09-13T09:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.045983221999999997 2024-09-13T09:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.951969+0000", 2024-09-13T09:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2442999999999998e-05 2024-09-13T09:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952032+0000", 2024-09-13T09:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999998e-05 2024-09-13T09:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.944113064, 2024-09-13T09:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:29.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:18:29.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:18:29.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:18:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:18:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:18:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:18:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:18:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:18:29.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:18:29.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:18:29.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:18:29.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.873 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.874 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.874 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:29.874 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.944095743, 2024-09-13T09:18:29.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:18:29.875 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.875 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.875 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:18:29.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.875 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:29.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:29.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:18:29.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:18:29.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:18:29.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:18:29.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:18:29.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:18:29.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:18:29.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.879 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.879 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:29.879 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.944078802, 2024-09-13T09:18:29.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:18:29.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.880 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:18:29.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.880 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:29.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:29.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:18:29.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:18:29.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:18:29.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:18:29.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:18:29.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:18:29.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:18:29.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:18:29.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:18:29.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:18:29.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:18:29.884 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.884 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.884 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.884 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.884 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:29.884 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.944061573999999, 2024-09-13T09:18:29.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:18:29.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.885 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.885 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.885 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.885 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.885 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:18:29.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.885 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:29.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:29.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:29.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:18:29.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:18:29.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:18:29.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:18:29.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:18:29.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:18:29.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:18:29.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:18:29.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.944025153, 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:18:29.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:18:29.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:29.896 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.943971691, 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.900 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.943949671, 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.943925036, 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:18:29.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.943906513, 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:29.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:18:29.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:18:29.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:18:29.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:18:29.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:18:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:18:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:18:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:18:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.943888641999999, 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:29.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.943871312000001, 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:18:29.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.943853841999999, 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:18:29.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:18:29.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:29.925 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.943836976, 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:29.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.929 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.943819188999999, 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.943774878999999, 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:18:29.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710644391000001, 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710627558000001, 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:18:29.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:18:29.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:18:29.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.710611200000001, 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:29.947 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:29.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:29.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:29.951 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:29.953 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:29.953 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:29.946+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:29.974 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-13T09:18:30.054 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:30.050+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:30.103 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:30.104 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:30.119 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:30.159 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-13T09:18:30.161 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:30.158+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:30.259 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:30.258+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:30.285 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:30.286 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:30.286 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:30.286 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:30.286 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:30.300 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:30.357 INFO:teuthology.orchestra.run.smithi032.stdout:3482 write 0x9c8ce11 thru 0x9c91b94 (0x4d84 bytes) 2024-09-13T09:18:30.361 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:30.358+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:30.431 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:30.431 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:30.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:30.431 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:30.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.431 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4388 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.431 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.443546771999998, 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069349509000000004, 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4388 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:30.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985769+0000", 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985775+0000", 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5230000000000001e-06 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985794+0000", 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8425999999999999e-05 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985809+0000", 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.486e-05 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.054965+0000", 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069156463000000001 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055036+0000", 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0780999999999999e-05 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055120+0000", 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3486000000000007e-05 2024-09-13T09:18:30.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4389 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.443496248999999, 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069473833999999998, 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4389 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985820+0000", 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985859+0000", 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9279999999999997e-05 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985881+0000", 2024-09-13T09:18:30.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2116999999999998e-05 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985896+0000", 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4698999999999999e-05 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055152+0000", 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069256568000000004 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055220+0000", 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8255000000000003e-05 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055294+0000", 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3858999999999999e-05 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4390 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:30.440 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.443400637000003, 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069563635999999998, 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4390 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985915+0000", 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985922+0000", 2024-09-13T09:18:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985941+0000", 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8260000000000001e-05 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985955+0000", 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4409999999999999e-05 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055325+0000", 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069370290000000001 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055395+0000", 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0158000000000002e-05 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055480+0000", 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4340999999999998e-05 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.443349570000002, 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:18:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:18:30.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:18:30.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:18:30.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.449 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.443300833999999, 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:18:30.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:18:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:18:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:18:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:18:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:18:30.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.443233165000002, 2024-09-13T09:18:30.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:18:30.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.443092956000001, 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:18:30.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:18:30.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:30.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:18:30.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:18:30.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:18:30.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.463 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.44300209, 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:18:30.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:18:30.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:18:30.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.468 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.468 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.468 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:30.468 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.442927230000002, 2024-09-13T09:18:30.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:18:30.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:18:30.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.469 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:30.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:18:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.442856054000003, 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.473 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:18:30.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:18:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:18:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:18:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:18:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:18:30.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.442807000999998, 2024-09-13T09:18:30.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:18:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:18:30.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.442745264999999, 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.482 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:18:30.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.442695595000004, 2024-09-13T09:18:30.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:18:30.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:18:30.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.442647076, 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:18:30.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:30.495 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.442598680000003, 2024-09-13T09:18:30.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:18:30.496 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.496 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.496 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.496 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.496 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.496 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:18:30.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.496 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:18:30.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:18:30.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.442551393000002, 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.442467991000001, 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:18:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.442394264999997, 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:18:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.442326141999999, 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.442275674999998, 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:18:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:18:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:18:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:30.524 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:30.525 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:30.522+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:30.625 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:30.622+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:30.656 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:30.656 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:30.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:30.762 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:30.758+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:30.827 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:30.822+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:30.844 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:30.845 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:30.859 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:30.945 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:30.942+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:31.029 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:31.026+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.341357199000001, 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.21372158699999999, 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685363+0000", 2024-09-13T09:18:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685372+0000", 2024-09-13T09:18:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4590000000000004e-06 2024-09-13T09:18:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685394+0000", 2024-09-13T09:18:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1616000000000001e-05 2024-09-13T09:18:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685407+0000", 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3675e-05 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685484+0000", 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7065999999999997e-05 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685493+0000", 2024-09-13T09:18:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8070000000000006e-06 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690770+0000", 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052763869999999996 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690940+0000", 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017001900000000001 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.691238+0000", 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029794499999999998 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.692057+0000", 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081943000000000001 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.898933+0000", 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.20687588200000001 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899026+0000", 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2777999999999998e-05 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899086+0000", 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0188e-05 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4523 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.988771+0000", 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037950553000002, 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037597541999999998, 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4523 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988771+0000", 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988771+0000", 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988770+0000", 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988774+0000", 2024-09-13T09:18:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4199999999999999e-06 2024-09-13T09:18:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988784+0000", 2024-09-13T09:18:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0463e-05 2024-09-13T09:18:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988791+0000", 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.026201+0000", 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037409501999999997 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.026273+0000", 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2779000000000002e-05 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.026368+0000", 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4963999999999994e-05 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4524 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.988800+0000", 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037921246000003, 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037827604000000001, 2024-09-13T09:18:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4524 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988800+0000", 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988800+0000", 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988800+0000", 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988815+0000", 2024-09-13T09:18:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5109999999999999e-05 2024-09-13T09:18:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988824+0000", 2024-09-13T09:18:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6490000000000003e-06 2024-09-13T09:18:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988831+0000", 2024-09-13T09:18:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:18:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.026437+0000", 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037605588000000002 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.026532+0000", 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5239000000000001e-05 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.026628+0000", 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5624000000000005e-05 2024-09-13T09:18:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4525 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.988840+0000", 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037881806999998, 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037977986999999998, 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4525 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988840+0000", 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988840+0000", 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988839+0000", 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988842+0000", 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3129999999999999e-06 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988854+0000", 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1177e-05 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988868+0000", 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4543e-05 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.026672+0000", 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037804108000000003 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.026736+0000", 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3411000000000007e-05 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.026818+0000", 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2058000000000006e-05 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4526 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.988880+0000", 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037841637, 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038323860000000001, 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4526 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988880+0000", 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988880+0000", 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988879+0000", 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988884+0000", 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3900000000000001e-06 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988901+0000", 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6566999999999999e-05 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988914+0000", 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2683e-05 2024-09-13T09:18:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.026872+0000", 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037958283000000002 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027025+0000", 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152707 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027204+0000", 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017911700000000001 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4527 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.988925+0000", 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037796616999998, 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038480522000000003, 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4527 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988925+0000", 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988925+0000", 2024-09-13T09:18:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988924+0000", 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988931+0000", 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.63e-06 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988948+0000", 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7207e-05 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988958+0000", 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0196e-05 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027255+0000", 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038297523999999999 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027319+0000", 2024-09-13T09:18:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3684999999999998e-05 2024-09-13T09:18:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027405+0000", 2024-09-13T09:18:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6234000000000001e-05 2024-09-13T09:18:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4528 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.989000+0000", 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037721382000001, 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038609908999999998, 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4528 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989000+0000", 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989000+0000", 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.988999+0000", 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989004+0000", 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0470000000000001e-06 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989017+0000", 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2466000000000001e-05 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989024+0000", 2024-09-13T09:18:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6299999999999998e-06 2024-09-13T09:18:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027449+0000", 2024-09-13T09:18:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038424694000000002 2024-09-13T09:18:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027522+0000", 2024-09-13T09:18:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2718000000000004e-05 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027610+0000", 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8317000000000002e-05 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4529 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.989041+0000", 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037680584999997, 2024-09-13T09:18:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038773889999999998, 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4529 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989041+0000", 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989041+0000", 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989040+0000", 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989044+0000", 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7500000000000001e-06 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989053+0000", 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5699999999999999e-06 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989060+0000", 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8170000000000001e-06 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027655+0000", 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038595034 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027725+0000", 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0091000000000004e-05 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027815+0000", 2024-09-13T09:18:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9578000000000002e-05 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4530 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.989066+0000", 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037655082000001, 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038991149000000003, 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4530 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989066+0000", 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989066+0000", 2024-09-13T09:18:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989066+0000", 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989069+0000", 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.236e-06 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989078+0000", 2024-09-13T09:18:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8969999999999997e-06 2024-09-13T09:18:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989085+0000", 2024-09-13T09:18:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.793e-06 2024-09-13T09:18:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027878+0000", 2024-09-13T09:18:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038793643000000003 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.027969+0000", 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0606999999999995e-05 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.028058+0000", 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8628999999999995e-05 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4531 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.989091+0000", 2024-09-13T09:18:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037630808999999, 2024-09-13T09:18:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039178614, 2024-09-13T09:18:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4531 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989091+0000", 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989091+0000", 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989090+0000", 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989093+0000", 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1099999999999999e-06 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989102+0000", 2024-09-13T09:18:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6829999999999997e-06 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989109+0000", 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.028103+0000", 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038994807999999999 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.028167+0000", 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3802999999999999e-05 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.028269+0000", 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4532 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.989114+0000", 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037607109, 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039480683000000003, 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4532 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989114+0000", 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989114+0000", 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989114+0000", 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989117+0000", 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8059999999999999e-06 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989125+0000", 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6940000000000001e-06 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989132+0000", 2024-09-13T09:18:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:18:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.028292+0000", 2024-09-13T09:18:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039159911999999998 2024-09-13T09:18:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.028418+0000", 2024-09-13T09:18:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012647 2024-09-13T09:18:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.028595+0000", 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017690100000000001 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4533 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.989138+0000", 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037583626, 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039659171999999999, 2024-09-13T09:18:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4533 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989138+0000", 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989138+0000", 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989137+0000", 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989140+0000", 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9869999999999999e-06 2024-09-13T09:18:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989149+0000", 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5660000000000007e-06 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989159+0000", 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.023e-05 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.028652+0000", 2024-09-13T09:18:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039493338000000003 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.028717+0000", 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4159999999999996e-05 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.028797+0000", 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0490999999999999e-05 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4534 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.989170+0000", 2024-09-13T09:18:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037551923000002, 2024-09-13T09:18:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039832204000000003, 2024-09-13T09:18:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4534 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989170+0000", 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989170+0000", 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989169+0000", 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989174+0000", 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3970000000000002e-06 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989190+0000", 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6325999999999999e-05 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989203+0000", 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.275e-05 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.028841+0000", 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039638257000000003 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.028914+0000", 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2717000000000002e-05 2024-09-13T09:18:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.029002+0000", 2024-09-13T09:18:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7609999999999996e-05 2024-09-13T09:18:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4664 3.16 3:6d1c421f:::rbd_data.1185192b47c4.0000000000000006:5 [read 741376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.989235+0000", 2024-09-13T09:18:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.037486319999999, 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039943183, 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4664 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989235+0000", 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989235+0000", 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989234+0000", 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989241+0000", 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5769999999999999e-06 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989261+0000", 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0293e-05 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.989275+0000", 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3923e-05 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.029037+0000", 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039761824000000001 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.029099+0000", 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1925000000000001e-05 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.029178+0000", 2024-09-13T09:18:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9671000000000003e-05 2024-09-13T09:18:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:14590 3.1d 3:be60f898:::rbd_data.11853ecd5f5f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13672152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.485508758999998, 2024-09-13T09:18:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052557382, 2024-09-13T09:18:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 14590 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541207+0000", 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562477+0000", 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021270155999999998 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562539+0000", 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1851000000000001e-05 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562565+0000", 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6194e-05 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562702+0000", 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013731299999999999 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.563071+0000", 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036892899999999999 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.565167+0000", 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020961959999999998 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.585261+0000", 2024-09-13T09:18:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020093443999999999 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593596+0000", 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083352849999999996 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593696+0000", 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100021 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593770+0000", 2024-09-13T09:18:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3889999999999999e-05 2024-09-13T09:18:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:14584 3.2 e39/22 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head v 39'64, mlcod=38'62)", 2024-09-13T09:18:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.483523112999997, 2024-09-13T09:18:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051546091000000002, 2024-09-13T09:18:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543197+0000", 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588691+0000", 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045493475999999998 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588711+0000", 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0485000000000001e-05 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588721+0000", 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.499e-06 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588766+0000", 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5228000000000001e-05 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588788+0000", 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2503000000000001e-05 2024-09-13T09:18:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594671+0000", 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058822809999999996 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594744+0000", 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3776000000000007e-05 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:22580 3.1c 3:39876c9f:::rbd_data.1185d6f8ec87.000000000000000f:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T09:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.174270323000002, 2024-09-13T09:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050372896, 2024-09-13T09:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22580 2024-09-13T09:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852449+0000", 2024-09-13T09:18:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852462+0000", 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3756e-05 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852495+0000", 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2435999999999999e-05 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852516+0000", 2024-09-13T09:18:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1347000000000001e-05 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852644+0000", 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012777200000000001 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852784+0000", 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013980500000000001 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.860936+0000", 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081526859999999993 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902522+0000", 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041585203000000001 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902560+0000", 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8603000000000003e-05 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902708+0000", 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014740900000000001 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902824+0000", 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116519 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:32875 3.5 3:a49b0d7c:::rbd_data.1185833655de.000000000000001b:16 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.887961+0000", 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.138760322, 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037721488999999997, 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 32875 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.887961+0000", 2024-09-13T09:18:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.887961+0000", 2024-09-13T09:18:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.887961+0000", 2024-09-13T09:18:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.887963+0000", 2024-09-13T09:18:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3130000000000001e-06 2024-09-13T09:18:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.887970+0000", 2024-09-13T09:18:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.887975+0000", 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6580000000000001e-06 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.925496+0000", 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037521693000000002 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.925586+0000", 2024-09-13T09:18:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9106999999999999e-05 2024-09-13T09:18:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.925683+0000", 2024-09-13T09:18:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7101000000000004e-05 2024-09-13T09:18:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:32876 3.5 3:a49b0d7c:::rbd_data.1185833655de.000000000000001b:16 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:16.887979+0000", 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.138742365000001, 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037955214000000001, 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 32876 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.887979+0000", 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.887979+0000", 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.887979+0000", 2024-09-13T09:18:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.887981+0000", 2024-09-13T09:18:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.238e-06 2024-09-13T09:18:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.887987+0000", 2024-09-13T09:18:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:18:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.887992+0000", 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5979999999999999e-06 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.925718+0000", 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037726216999999999 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.925809+0000", 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0476999999999994e-05 2024-09-13T09:18:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:16.925934+0000", 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012566500000000001 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:48768 3.1d 3:b963585c:::rbd_data.118519085f7e.0000000000000005:17 [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:24.395669+0000", 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.6310521160000002, 2024-09-13T09:18:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039698984, 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 48768 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:24.395669+0000", 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:24.395669+0000", 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:24.395668+0000", 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:24.395674+0000", 2024-09-13T09:18:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6239999999999999e-06 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:24.395691+0000", 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6782999999999998e-05 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:24.395704+0000", 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2799000000000001e-05 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:24.427363+0000", 2024-09-13T09:18:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031659731000000003 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:24.427557+0000", 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000193635 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:24.435368+0000", 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078113219999999999 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:31.129 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:31.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:18:31.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:31.131 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:31.126+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:31.185 INFO:teuthology.orchestra.run.smithi032.stdout:3483 read 0x98e51bd thru 0x98ee9dd (0x9821 bytes) 2024-09-13T09:18:31.185 INFO:teuthology.orchestra.run.smithi032.stdout:3486 read 0x34568db thru 0x345a57a (0x3ca0 bytes) 2024-09-13T09:18:31.186 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:31.186 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:31.199 INFO:teuthology.orchestra.run.smithi032.stdout:3490 write 0xe8e902b thru 0xe8f600d (0xcfe3 bytes) 2024-09-13T09:18:31.205 INFO:teuthology.orchestra.run.smithi032.stdout:3491 write 0x5409631 thru 0x5417130 (0xdb00 bytes) 2024-09-13T09:18:31.213 INFO:teuthology.orchestra.run.smithi032.stdout:3492 write 0x243bece thru 0x244970b (0xd83e bytes) 2024-09-13T09:18:31.231 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:31.226+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:31.240 INFO:teuthology.orchestra.run.smithi032.stdout:3493 trunc from 0xe8f600e to 0x38aa128 2024-09-13T09:18:31.263 INFO:teuthology.orchestra.run.smithi032.stdout:3494 read 0x27fe253 thru 0x2803589 (0x5337 bytes) 2024-09-13T09:18:31.264 INFO:teuthology.orchestra.run.smithi032.stdout:3495 read 0x2dab804 thru 0x2db47c1 (0x8fbe bytes) 2024-09-13T09:18:31.265 INFO:teuthology.orchestra.run.smithi032.stdout:3496 read 0x2e0e204 thru 0x2e16923 (0x8720 bytes) 2024-09-13T09:18:31.265 INFO:teuthology.orchestra.run.smithi032.stdout:3497 write 0x143f480 thru 0x1445c41 (0x67c2 bytes) 2024-09-13T09:18:31.280 INFO:teuthology.orchestra.run.smithi032.stdout:3498 punch from 0x2524acc to 0x2526645, (0x1b79 bytes) 2024-09-13T09:18:31.280 INFO:teuthology.orchestra.run.smithi032.stdout:3499 read 0x1ef4660 thru 0x1f03e17 (0xf7b8 bytes) 2024-09-13T09:18:31.280 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:31.281 INFO:teuthology.orchestra.run.smithi032.stdout:3501 read 0xfd49ba thru 0xfdc565 (0x7bac bytes) 2024-09-13T09:18:31.281 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:31.281 INFO:teuthology.orchestra.run.smithi032.stdout:3504 write 0x152c948 thru 0x152ece3 (0x239c bytes) 2024-09-13T09:18:31.286 INFO:teuthology.orchestra.run.smithi032.stdout:3506 read 0x21e7426 thru 0x21ed61f (0x61fa bytes) 2024-09-13T09:18:31.287 INFO:teuthology.orchestra.run.smithi032.stdout:3507 write 0x212e477 thru 0x2131849 (0x33d3 bytes) 2024-09-13T09:18:31.300 INFO:teuthology.orchestra.run.smithi032.stdout:3509 read 0x2393705 thru 0x2398425 (0x4d21 bytes) 2024-09-13T09:18:31.304 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:31.305 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:31.315 INFO:teuthology.orchestra.run.smithi032.stdout:3510 write 0x67d99fb thru 0x67dae8d (0x1493 bytes) 2024-09-13T09:18:31.315 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:31.315 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:31.318 INFO:teuthology.orchestra.run.smithi032.stdout:3511 write 0x14be203 thru 0x14cbbb6 (0xd9b4 bytes) 2024-09-13T09:18:31.321 INFO:teuthology.orchestra.run.smithi032.stdout:3512 read 0x2b456e8 thru 0x2b4e3ea (0x8d03 bytes) 2024-09-13T09:18:31.322 INFO:teuthology.orchestra.run.smithi032.stdout:3514 read 0x573c317 thru 0x573cd51 (0xa3b bytes) 2024-09-13T09:18:31.322 INFO:teuthology.orchestra.run.smithi032.stdout:3515 read 0x5090b20 thru 0x509d854 (0xcd35 bytes) 2024-09-13T09:18:31.323 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:31.323 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:31.330 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:31.340 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:31.341 INFO:teuthology.orchestra.run.smithi032.stdout:3517 writesame 0xdba354f thru 0xdba56ab data_size 0xd(0x215d bytes) 2024-09-13T09:18:31.341 INFO:teuthology.orchestra.run.smithi032.stdout:3519 read 0xb61a469 thru 0xb628f6c (0xeb04 bytes) 2024-09-13T09:18:31.341 INFO:teuthology.orchestra.run.smithi032.stdout:3520 write 0x6402125 thru 0x640e3b2 (0xc28e bytes) 2024-09-13T09:18:31.344 INFO:teuthology.orchestra.run.smithi032.stdout:3521 writesame 0xb5e9550 thru 0xb5f037c data_size 0x1(0x6e2d bytes) 2024-09-13T09:18:31.360 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:31.361 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:31.361 INFO:teuthology.orchestra.run.smithi032.stdout:3522 read 0x52c289d thru 0x52c2ad9 (0x23d bytes) 2024-09-13T09:18:31.361 INFO:teuthology.orchestra.run.smithi032.stdout:3523 read 0x2fd5524 thru 0x2fdda29 (0x8506 bytes) 2024-09-13T09:18:31.374 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:31.370+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:31.375 INFO:teuthology.orchestra.run.smithi032.stdout:3524 trunc from 0xdba56ac to 0x3a9853e 2024-09-13T09:18:31.380 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:31.393 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:31.398 INFO:teuthology.orchestra.run.smithi032.stdout:3526 trunc from 0x3a9853e to 0x72cc49b 2024-09-13T09:18:31.402 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:31.405 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:31.405 INFO:teuthology.orchestra.run.smithi032.stdout:3528 punch from 0x2ccbf6d to 0x2cd239a, (0x642d bytes) 2024-09-13T09:18:31.405 INFO:teuthology.orchestra.run.smithi032.stdout:3529 read 0x2946631 thru 0x29563ae (0xfd7e bytes) 2024-09-13T09:18:31.406 INFO:teuthology.orchestra.run.smithi032.stdout:3530 write 0x3fbfef3 thru 0x3fcbdc9 (0xbed7 bytes) 2024-09-13T09:18:31.409 INFO:teuthology.orchestra.run.smithi032.stdout:3531 punch from 0x4629e2c to 0x462dbfa, (0x3dce bytes) 2024-09-13T09:18:31.409 INFO:teuthology.orchestra.run.smithi032.stdout:3532 read 0x1b5cf2a thru 0x1b5f422 (0x24f9 bytes) 2024-09-13T09:18:31.422 INFO:teuthology.orchestra.run.smithi032.stdout:3533 writesame 0xb250583 thru 0xb2562e3 data_size 0x7(0x5d61 bytes) 2024-09-13T09:18:31.424 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:31.425 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:31.432 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:31.430+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:31.433 INFO:teuthology.orchestra.run.smithi032.stdout:3534 trunc from 0xb2562e4 to 0x2ed38bc 2024-09-13T09:18:31.444 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:31.445 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:31.463 INFO:teuthology.orchestra.run.smithi032.stdout:3535 clone 25 order 24 su 8388608 sc 5 2024-09-13T09:18:31.513 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:31.514 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:31.527 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:31.573 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:31.570+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:31.634 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:31.630+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:31.654 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:31.654 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:31.668 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:31.735 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:31.734+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:31.836 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:31.830+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:31.838 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:31.838 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:31.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:31.937 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:31.934+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:32.022 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:32.023 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:32.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:32.038 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:32.034+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:32.140 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:32.138+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:32.157 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:32.157 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:32.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:32.176 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:18:32.176 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:18:32.177 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-13T09:18:32.177 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:18:32.177 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:18:32.177 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-13T09:18:32.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:18:32.263 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:32.262+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:32.277 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone24 from 0x2ed38bc (overlap 0x2ed38bc) to 0x2a24ca9 2024-09-13T09:18:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:32.306 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:18:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:32.330 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:32.330 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:32.341 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:32.338+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:32.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:32.405 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-13T09:18:32.442 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:32.438+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:32.513 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:32.513 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:32.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:32.530 INFO:teuthology.orchestra.run.smithi032.stdout:3537 trunc from 0x2ed38bc to 0x478e17a 2024-09-13T09:18:32.543 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:32.538+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:32.554 INFO:teuthology.orchestra.run.smithi032.stdout:3538 write 0x5fa0295 thru 0x5fa7b3d (0x78a9 bytes) 2024-09-13T09:18:32.562 INFO:teuthology.orchestra.run.smithi032.stdout:3539 write 0x4f72b2e thru 0x4f7d958 (0xae2b bytes) 2024-09-13T09:18:32.571 INFO:teuthology.orchestra.run.smithi032.stdout:3540 writesame 0x39fb0a8 thru 0x3a0af73 data_size 0x2e(0xfecc bytes) 2024-09-13T09:18:32.588 INFO:teuthology.orchestra.run.smithi032.stdout:3541 read 0x4ffe7c7 thru 0x500632b (0x7b65 bytes) 2024-09-13T09:18:32.589 INFO:teuthology.orchestra.run.smithi032.stdout:3542 read 0x43b1ba4 thru 0x43bed23 (0xd180 bytes) 2024-09-13T09:18:32.615 INFO:teuthology.orchestra.run.smithi032.stdout:3543 writesame 0xe07d728 thru 0xe07eb0b data_size 0x13(0x13e4 bytes) 2024-09-13T09:18:32.615 INFO:teuthology.orchestra.run.smithi032.stdout:3544 read 0x9a4e7dc thru 0x9a52590 (0x3db5 bytes) 2024-09-13T09:18:32.615 INFO:teuthology.orchestra.run.smithi032.stdout:3545 writesame 0x5dfa2e3 thru 0x5e05759 data_size 0x1(0xb477 bytes) 2024-09-13T09:18:32.644 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:32.642+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:32.659 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:32.660 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:32.673 INFO:teuthology.orchestra.run.smithi032.stdout:3546 read 0xbab7bdc thru 0xbabf0c1 (0x74e6 bytes) 2024-09-13T09:18:32.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:32.675 INFO:teuthology.orchestra.run.smithi032.stdout:3547 writesame 0x8dd5ac0 thru 0x8ddcced data_size 0xa(0x722e bytes) 2024-09-13T09:18:32.683 INFO:teuthology.orchestra.run.smithi032.stdout:3548 flatten 2024-09-13T09:18:32.727 INFO:teuthology.orchestra.run.smithi032.stdout:3549 writesame 0x5bea993 thru 0x5bf6d33 data_size 0x1(0xc3a1 bytes) 2024-09-13T09:18:32.745 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:32.738+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:32.790 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:32.790 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:32.792 INFO:teuthology.orchestra.run.smithi032.stdout:3552 trunc from 0xe07eb0c to 0xcaddb3c 2024-09-13T09:18:32.806 INFO:teuthology.orchestra.run.smithi032.stdout:3554 read 0xb4db000 thru 0xb4e3e89 (0x8e8a bytes) 2024-09-13T09:18:32.806 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:32.806 INFO:teuthology.orchestra.run.smithi032.stdout:3556 write 0xed6049 thru 0xedc26d (0x6225 bytes) 2024-09-13T09:18:32.809 INFO:teuthology.orchestra.run.smithi032.stdout:3557 read 0x55233e7 thru 0x5527439 (0x4053 bytes) 2024-09-13T09:18:32.818 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:32.818 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:32.820 INFO:teuthology.orchestra.run.smithi032.stdout:3559 trunc from 0xcaddb3c to 0x7b3f084 2024-09-13T09:18:32.834 INFO:teuthology.orchestra.run.smithi032.stdout:3560 write 0x6f9a80d thru 0x6fa8cc9 (0xe4bd bytes) 2024-09-13T09:18:32.836 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:32.841 INFO:teuthology.orchestra.run.smithi032.stdout:3561 write 0x53053c3 thru 0x530c958 (0x7596 bytes) 2024-09-13T09:18:32.865 INFO:teuthology.orchestra.run.smithi032.stdout:3562 writesame 0xe22c2fe thru 0xe2324b1 data_size 0x4a(0x61b4 bytes) 2024-09-13T09:18:32.866 INFO:teuthology.orchestra.run.smithi032.stdout:3564 write 0xdfc0c53 thru 0xdfce5ae (0xd95c bytes) 2024-09-13T09:18:32.866 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:32.862+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:32.871 INFO:teuthology.orchestra.run.smithi032.stdout:3565 write 0xd353210 thru 0xd354cbc (0x1aad bytes) 2024-09-13T09:18:32.876 INFO:teuthology.orchestra.run.smithi032.stdout:3566 read 0xa496bdd thru 0xa49c143 (0x5567 bytes) 2024-09-13T09:18:32.885 INFO:teuthology.orchestra.run.smithi032.stdout:3568 write 0xe3f302c thru 0xe3f8895 (0x586a bytes) 2024-09-13T09:18:32.892 INFO:teuthology.orchestra.run.smithi032.stdout:3569 read 0xc76edc thru 0xc77c34 (0xd59 bytes) 2024-09-13T09:18:32.892 INFO:teuthology.orchestra.run.smithi032.stdout:3570 read 0x534f607 thru 0x53545e9 (0x4fe3 bytes) 2024-09-13T09:18:32.893 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:32.913 INFO:teuthology.orchestra.run.smithi032.stdout:3572 trunc from 0xe3f8896 to 0xd86c34 2024-09-13T09:18:32.946 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:32.942+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:32.949 INFO:teuthology.orchestra.run.smithi032.stdout:3574 read 0xbe5806 thru 0xbe71e0 (0x19db bytes) 2024-09-13T09:18:32.949 INFO:teuthology.orchestra.run.smithi032.stdout:3575 read 0x3a84bf thru 0x3b72f2 (0xee34 bytes) 2024-09-13T09:18:32.972 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:32.972 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:32.976 INFO:teuthology.orchestra.run.smithi032.stdout:3576 write 0x7c20d31 thru 0x7c24ba0 (0x3e70 bytes) 2024-09-13T09:18:32.983 INFO:teuthology.orchestra.run.smithi032.stdout:3577 punch from 0x2bc2a79 to 0x2bc619b, (0x3722 bytes) 2024-09-13T09:18:32.983 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:32.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:32.991 INFO:teuthology.orchestra.run.smithi032.stdout:3579 trunc from 0x7c24ba1 to 0xb07c258 2024-09-13T09:18:33.001 INFO:teuthology.orchestra.run.smithi032.stdout:3583 write 0xa62a655 thru 0xa635762 (0xb10e bytes) 2024-09-13T09:18:33.009 INFO:teuthology.orchestra.run.smithi032.stdout:3584 read 0x3a15dfb thru 0x3a18394 (0x259a bytes) 2024-09-13T09:18:33.009 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.009 INFO:teuthology.orchestra.run.smithi032.stdout:3588 write 0x3ca4a19 thru 0x3cae551 (0x9b39 bytes) 2024-09-13T09:18:33.015 INFO:teuthology.orchestra.run.smithi032.stdout:3589 read 0x771e63 thru 0x77a19a (0x8338 bytes) 2024-09-13T09:18:33.016 INFO:teuthology.orchestra.run.smithi032.stdout:3590 writesame 0x3cd5837 thru 0x3ce1198 data_size 0x2(0xb962 bytes) 2024-09-13T09:18:33.047 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:33.042+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:33.048 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.049 INFO:teuthology.orchestra.run.smithi032.stdout:3594 write 0x7504dca thru 0x750d698 (0x88cf bytes) 2024-09-13T09:18:33.055 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.055 INFO:teuthology.orchestra.run.smithi032.stdout:3597 read 0x1962d59 thru 0x196e9d6 (0xbc7e bytes) 2024-09-13T09:18:33.055 INFO:teuthology.orchestra.run.smithi032.stdout:3599 writesame 0x991bde6 thru 0x991d641 data_size 0x2(0x185c bytes) 2024-09-13T09:18:33.059 INFO:teuthology.orchestra.run.smithi032.stdout:3600 write 0x904e56d thru 0x905e065 (0xfaf9 bytes) 2024-09-13T09:18:33.066 INFO:teuthology.orchestra.run.smithi032.stdout:3601 write 0x971a5b9 thru 0x9729df2 (0xf83a bytes) 2024-09-13T09:18:33.078 INFO:teuthology.orchestra.run.smithi032.stdout:3603 trunc from 0xb07c258 to 0xe2d7954 2024-09-13T09:18:33.090 INFO:teuthology.orchestra.run.smithi032.stdout:3604 read 0xdc5af2d thru 0xdc645db (0x96af bytes) 2024-09-13T09:18:33.091 INFO:teuthology.orchestra.run.smithi032.stdout:3605 read 0xcf553f2 thru 0xcf5e5e8 (0x91f7 bytes) 2024-09-13T09:18:33.091 INFO:teuthology.orchestra.run.smithi032.stdout:3606 read 0x3848016 thru 0x384ae73 (0x2e5e bytes) 2024-09-13T09:18:33.092 INFO:teuthology.orchestra.run.smithi032.stdout:3607 write 0x9b03ca2 thru 0x9b07fd1 (0x4330 bytes) 2024-09-13T09:18:33.096 INFO:teuthology.orchestra.run.smithi032.stdout:3608 write 0x183dcf0 thru 0x1844b62 (0x6e73 bytes) 2024-09-13T09:18:33.103 INFO:teuthology.orchestra.run.smithi032.stdout:3609 read 0x455a622 thru 0x456a38b (0xfd6a bytes) 2024-09-13T09:18:33.103 INFO:teuthology.orchestra.run.smithi032.stdout:3610 write 0xa010f56 thru 0xa0148a7 (0x3952 bytes) 2024-09-13T09:18:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:3611 write 0x1590b83 thru 0x1597b8f (0x700d bytes) 2024-09-13T09:18:33.111 INFO:teuthology.orchestra.run.smithi032.stdout:3612 write 0x1d5a9f3 thru 0x1d5e526 (0x3b34 bytes) 2024-09-13T09:18:33.123 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:33.123 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:33.143 INFO:teuthology.orchestra.run.smithi032.stdout:3613 trunc from 0xe2d7954 to 0x1d73604 2024-09-13T09:18:33.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:33.148 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:33.142+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:33.177 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:18:33.201 INFO:teuthology.orchestra.run.smithi032.stdout:3614 write 0xb8dc0c2 thru 0xb8dce7a (0xdb9 bytes) 2024-09-13T09:18:33.209 INFO:teuthology.orchestra.run.smithi032.stdout:3615 write 0x1b2e601 thru 0x1b370e0 (0x8ae0 bytes) 2024-09-13T09:18:33.213 INFO:teuthology.orchestra.run.smithi032.stdout:3616 write 0x87a9f2d thru 0x87aa3ef (0x4c3 bytes) 2024-09-13T09:18:33.220 INFO:teuthology.orchestra.run.smithi032.stdout:3617 read 0x55f0204 thru 0x55f7aa0 (0x789d bytes) 2024-09-13T09:18:33.220 INFO:teuthology.orchestra.run.smithi032.stdout:3619 read 0xb3f3abd thru 0xb3f496b (0xeaf bytes) 2024-09-13T09:18:33.224 INFO:teuthology.orchestra.run.smithi032.stdout:3620 trunc from 0xb8dce7b to 0xdd28343 2024-09-13T09:18:33.232 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.232 INFO:teuthology.orchestra.run.smithi032.stdout:3623 punch from 0x7468bc to 0x746d2e, (0x472 bytes) 2024-09-13T09:18:33.232 INFO:teuthology.orchestra.run.smithi032.stdout:3624 read 0x71605a8 thru 0x7162295 (0x1cee bytes) 2024-09-13T09:18:33.232 INFO:teuthology.orchestra.run.smithi032.stdout:3625 write 0x90e860b thru 0x90f1b8a (0x9580 bytes) 2024-09-13T09:18:33.239 INFO:teuthology.orchestra.run.smithi032.stdout:3626 punch from 0x7e67c9 to 0x7ecd7e, (0x65b5 bytes) 2024-09-13T09:18:33.239 INFO:teuthology.orchestra.run.smithi032.stdout:3629 read 0x900850 thru 0x90ff91 (0xf742 bytes) 2024-09-13T09:18:33.240 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.240 INFO:teuthology.orchestra.run.smithi032.stdout:3635 write 0x1388938 thru 0x138e0a3 (0x576c bytes) 2024-09-13T09:18:33.247 INFO:teuthology.orchestra.run.smithi032.stdout:3636 trunc from 0xdd28343 to 0xbe48b51 2024-09-13T09:18:33.249 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:33.246+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:33.260 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.260 INFO:teuthology.orchestra.run.smithi032.stdout:3639 write 0xa279314 thru 0xa289069 (0xfd56 bytes) 2024-09-13T09:18:33.267 INFO:teuthology.orchestra.run.smithi032.stdout:3640 write 0x7105d86 thru 0x7107734 (0x19af bytes) 2024-09-13T09:18:33.273 INFO:teuthology.orchestra.run.smithi032.stdout:3641 punch from 0xbbf6f12 to 0xbbf9619, (0x2707 bytes) 2024-09-13T09:18:33.273 INFO:teuthology.orchestra.run.smithi032.stdout:3642 write 0x6ba69e4 thru 0x6baad1b (0x4338 bytes) 2024-09-13T09:18:33.276 INFO:teuthology.orchestra.run.smithi032.stdout:3644 writesame 0x3e549e6 thru 0x3e59b0c data_size 0x19(0x5127 bytes) 2024-09-13T09:18:33.280 INFO:teuthology.orchestra.run.smithi032.stdout:3646 punch from 0x9965e04 to 0x996f94b, (0x9b47 bytes) 2024-09-13T09:18:33.280 INFO:teuthology.orchestra.run.smithi032.stdout:3647 write 0x6ace3ee thru 0x6ad8380 (0x9f93 bytes) 2024-09-13T09:18:33.295 INFO:teuthology.orchestra.run.smithi032.stdout:3648 trunc from 0xbe48b51 to 0x4bfe99c 2024-09-13T09:18:33.319 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:33.319 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:33.329 INFO:teuthology.orchestra.run.smithi032.stdout:3649 write 0x4e91590 thru 0x4e9dfd7 (0xca48 bytes) 2024-09-13T09:18:33.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:33.336 INFO:teuthology.orchestra.run.smithi032.stdout:3650 read 0x43805e7 thru 0x4386e7c (0x6896 bytes) 2024-09-13T09:18:33.337 INFO:teuthology.orchestra.run.smithi032.stdout:3651 punch from 0x1387cbf to 0x138e3ff, (0x6740 bytes) 2024-09-13T09:18:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:3652 trunc from 0x4e9dfd8 to 0x35ac0fe 2024-09-13T09:18:33.350 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:33.346+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:33.361 INFO:teuthology.orchestra.run.smithi032.stdout:3653 writesame 0x3a04f51 thru 0x3a14620 data_size 0x58(0xf6d0 bytes) 2024-09-13T09:18:33.378 INFO:teuthology.orchestra.run.smithi032.stdout:3654 trunc from 0x3a14621 to 0xc48f2e1 2024-09-13T09:18:33.396 INFO:teuthology.orchestra.run.smithi032.stdout:3655 trunc from 0xc48f2e1 to 0x62d3d8e 2024-09-13T09:18:33.405 INFO:teuthology.orchestra.run.smithi032.stdout:3656 read 0x26c09f0 thru 0x26c759b (0x6bac bytes) 2024-09-13T09:18:33.406 INFO:teuthology.orchestra.run.smithi032.stdout:3658 read 0x1c82fd3 thru 0x1c8faa3 (0xcad1 bytes) 2024-09-13T09:18:33.407 INFO:teuthology.orchestra.run.smithi032.stdout:3659 writesame 0x4e957ac thru 0x4e9cfbc data_size 0x1(0x7811 bytes) 2024-09-13T09:18:33.452 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:33.450+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:33.454 INFO:teuthology.orchestra.run.smithi032.stdout:3660 trunc from 0x62d3d8e to 0x30e4852 2024-09-13T09:18:33.466 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:33.466 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:33.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:33.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:33.502 INFO:teuthology.orchestra.run.smithi032.stdout:3662 write 0xc4a369c thru 0xc4a80fe (0x4a63 bytes) 2024-09-13T09:18:33.509 INFO:teuthology.orchestra.run.smithi032.stdout:3664 read 0x5f8bacf thru 0x5f97d76 (0xc2a8 bytes) 2024-09-13T09:18:33.510 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.510 INFO:teuthology.orchestra.run.smithi032.stdout:3667 read 0x7d96116 thru 0x7d9955b (0x3446 bytes) 2024-09-13T09:18:33.510 INFO:teuthology.orchestra.run.smithi032.stdout:3668 read 0x40356f3 thru 0x40452a7 (0xfbb5 bytes) 2024-09-13T09:18:33.513 INFO:teuthology.orchestra.run.smithi032.stdout:3670 trunc from 0xc4a80ff to 0xa3bec6f 2024-09-13T09:18:33.532 INFO:teuthology.orchestra.run.smithi032.stdout:3671 read 0x556c195 thru 0x5577ec1 (0xbd2d bytes) 2024-09-13T09:18:33.542 INFO:teuthology.orchestra.run.smithi032.stdout:3674 trunc from 0xa3bec6f to 0x40374a8 2024-09-13T09:18:33.552 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:33.546+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:33.554 INFO:teuthology.orchestra.run.smithi032.stdout:3675 read 0x3f358e7 thru 0x3f3db39 (0x8253 bytes) 2024-09-13T09:18:33.554 INFO:teuthology.orchestra.run.smithi032.stdout:3676 read 0x2f2bc8e thru 0x2f350a8 (0x941b bytes) 2024-09-13T09:18:33.556 INFO:teuthology.orchestra.run.smithi032.stdout:3677 trunc from 0x40374a8 to 0x49be3a1 2024-09-13T09:18:33.565 INFO:teuthology.orchestra.run.smithi032.stdout:3678 punch from 0x24706c2 to 0x247d315, (0xcc53 bytes) 2024-09-13T09:18:33.565 INFO:teuthology.orchestra.run.smithi032.stdout:3679 punch from 0x42cebe4 to 0x42d4742, (0x5b5e bytes) 2024-09-13T09:18:33.581 INFO:teuthology.orchestra.run.smithi032.stdout:3680 writesame 0x900af3f thru 0x900be0f data_size 0x1(0xed1 bytes) 2024-09-13T09:18:33.581 INFO:teuthology.orchestra.run.smithi032.stdout:3681 read 0x42431bb thru 0x4244b28 (0x196e bytes) 2024-09-13T09:18:33.582 INFO:teuthology.orchestra.run.smithi032.stdout:3682 write 0x127ce3b thru 0x127fbe4 (0x2daa bytes) 2024-09-13T09:18:33.596 INFO:teuthology.orchestra.run.smithi032.stdout:3684 trunc from 0x900be10 to 0xe9ecdb4 2024-09-13T09:18:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:3685 punch from 0x179b770 to 0x179d267, (0x1af7 bytes) 2024-09-13T09:18:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:3686 read 0x677f969 thru 0x678f71f (0xfdb7 bytes) 2024-09-13T09:18:33.606 INFO:teuthology.orchestra.run.smithi032.stdout:3688 write 0x857c500 thru 0x8588591 (0xc092 bytes) 2024-09-13T09:18:33.613 INFO:teuthology.orchestra.run.smithi032.stdout:3689 read 0x23ea3e2 thru 0x23f8f5e (0xeb7d bytes) 2024-09-13T09:18:33.614 INFO:teuthology.orchestra.run.smithi032.stdout:3690 punch from 0x936eddf to 0x9370556, (0x1777 bytes) 2024-09-13T09:18:33.614 INFO:teuthology.orchestra.run.smithi032.stdout:3691 read 0x4ff5ff2 thru 0x4ffec2c (0x8c3b bytes) 2024-09-13T09:18:33.615 INFO:teuthology.orchestra.run.smithi032.stdout:3693 read 0x3dec1d8 thru 0x3df2282 (0x60ab bytes) 2024-09-13T09:18:33.616 INFO:teuthology.orchestra.run.smithi032.stdout:3695 punch from 0xae28cca to 0xae2df6a, (0x52a0 bytes) 2024-09-13T09:18:33.616 INFO:teuthology.orchestra.run.smithi032.stdout:3696 read 0xa7487e1 thru 0xa74b968 (0x3188 bytes) 2024-09-13T09:18:33.616 INFO:teuthology.orchestra.run.smithi032.stdout:3697 punch from 0x96a2ab5 to 0x96b14e7, (0xea32 bytes) 2024-09-13T09:18:33.616 INFO:teuthology.orchestra.run.smithi032.stdout:3698 write 0xb5e0bb7 thru 0xb5e1fb5 (0x13ff bytes) 2024-09-13T09:18:33.622 INFO:teuthology.orchestra.run.smithi032.stdout:3699 write 0xa738247 thru 0xa738a1f (0x7d9 bytes) 2024-09-13T09:18:33.627 INFO:teuthology.orchestra.run.smithi032.stdout:3700 read 0xb71156 thru 0xb7fe1c (0xecc7 bytes) 2024-09-13T09:18:33.628 INFO:teuthology.orchestra.run.smithi032.stdout:3705 punch from 0xe6c627d to 0xe6c8edf, (0x2c62 bytes) 2024-09-13T09:18:33.628 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.628 INFO:teuthology.orchestra.run.smithi032.stdout:3711 punch from 0xcb8e58b to 0xcb96fe3, (0x8a58 bytes) 2024-09-13T09:18:33.628 INFO:teuthology.orchestra.run.smithi032.stdout:3712 punch from 0xaa932ff to 0xaa939e7, (0x6e8 bytes) 2024-09-13T09:18:33.628 INFO:teuthology.orchestra.run.smithi032.stdout:3715 writesame 0x58ae6bc thru 0x58b96c8 data_size 0x53(0xb00d bytes) 2024-09-13T09:18:33.631 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.637 INFO:teuthology.orchestra.run.smithi032.stdout:3719 trunc from 0xe9ecdb4 to 0xa5cc1f3 2024-09-13T09:18:33.647 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:33.647 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:33.653 INFO:teuthology.orchestra.run.smithi032.stdout:3721 write 0x6a120ee thru 0x6a1a078 (0x7f8b bytes) 2024-09-13T09:18:33.653 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:33.650+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:33.655 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:33.656 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:33.659 INFO:teuthology.orchestra.run.smithi032.stdout:3723 writesame 0x37d471f thru 0x37e13d0 data_size 0x85(0xccb2 bytes) 2024-09-13T09:18:33.663 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.663 INFO:teuthology.orchestra.run.smithi032.stdout:3725 writesame 0xda42c2 thru 0xdaaa74 data_size 0x1(0x67b3 bytes) 2024-09-13T09:18:33.666 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:33.674 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:33.674 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:33.677 INFO:teuthology.orchestra.run.smithi032.stdout:3726 write 0x9fe20f4 thru 0x9fe3202 (0x110f bytes) 2024-09-13T09:18:33.682 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.696 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:33.697 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:33.697 INFO:teuthology.orchestra.run.smithi032.stdout:3728 writesame 0xe78f194 thru 0xe79c833 data_size 0x44(0xd6a0 bytes) 2024-09-13T09:18:33.697 INFO:teuthology.orchestra.run.smithi032.stdout:3729 read 0xd20927 thru 0xd22c32 (0x230c bytes) 2024-09-13T09:18:33.698 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.698 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.698 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.698 INFO:teuthology.orchestra.run.smithi032.stdout:3733 write 0x2909fd6 thru 0x290ebc0 (0x4beb bytes) 2024-09-13T09:18:33.700 INFO:teuthology.orchestra.run.smithi032.stdout:3734 punch from 0xde7ca9c to 0xde82fbd, (0x6521 bytes) 2024-09-13T09:18:33.700 INFO:teuthology.orchestra.run.smithi032.stdout:3735 punch from 0x54535d6 to 0x5459127, (0x5b51 bytes) 2024-09-13T09:18:33.700 INFO:teuthology.orchestra.run.smithi032.stdout:3736 read 0x1282463 thru 0x1285999 (0x3537 bytes) 2024-09-13T09:18:33.700 INFO:teuthology.orchestra.run.smithi032.stdout:3737 write 0x26501d5 thru 0x265d876 (0xd6a2 bytes) 2024-09-13T09:18:33.704 INFO:teuthology.orchestra.run.smithi032.stdout:3738 write 0xbb6c27d thru 0xbb6e6b5 (0x2439 bytes) 2024-09-13T09:18:33.714 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:33.730 INFO:teuthology.orchestra.run.smithi032.stdout:3739 trunc from 0xe79c834 to 0xefcec9 2024-09-13T09:18:33.736 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:33.737 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:33.754 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:33.750+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:33.760 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:33.786 INFO:teuthology.orchestra.run.smithi032.stdout:3740 writesame 0x9a1a423 thru 0x9a294c4 data_size 0x3(0xf0a2 bytes) 2024-09-13T09:18:33.786 INFO:teuthology.orchestra.run.smithi032.stdout:3742 read 0x8da27c0 thru 0x8da6d4d (0x458e bytes) 2024-09-13T09:18:33.786 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:33.786 INFO:teuthology.orchestra.run.smithi032.stdout:3744 write 0xd502 thru 0xe078 (0xb77 bytes) 2024-09-13T09:18:33.790 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:33.791 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:33.803 INFO:teuthology.orchestra.run.smithi032.stdout:3745 write 0xea9903a thru 0xea9f772 (0x6739 bytes) 2024-09-13T09:18:33.805 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51073 3.1e 3:7a90b897:::rbd_object_map.118568665621:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:18:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:33.804953+0000", 2024-09-13T09:18:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.00085732799999999997, 2024-09-13T09:18:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00087817799999999997, 2024-09-13T09:18:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51073 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:33.804953+0000", 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:33.804953+0000", 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:33.804952+0000", 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:33.804958+0000", 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9839999999999998e-06 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:33.804972+0000", 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3988000000000001e-05 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:33.804980+0000", 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7340000000000008e-06 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:33.805021+0000", 2024-09-13T09:18:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0847999999999999e-05 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:33.805080+0000", 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9379000000000002e-05 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:33.805291+0000", 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021115399999999999 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:18:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 1 2024-09-13T09:18:33.810 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:33.811 INFO:teuthology.orchestra.run.smithi032.stdout:3747 punch from 0x18dc8cf to 0x18e0c48, (0x4379 bytes) 2024-09-13T09:18:33.811 INFO:teuthology.orchestra.run.smithi032.stdout:3748 read 0x1e9e51c thru 0x1e9f19b (0xc80 bytes) 2024-09-13T09:18:33.811 INFO:teuthology.orchestra.run.smithi032.stdout:3749 write 0x16a0384 thru 0x16ac984 (0xc601 bytes) 2024-09-13T09:18:33.813 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:33.817 INFO:teuthology.orchestra.run.smithi032.stdout:3750 write 0x272bb77 thru 0x2736427 (0xa8b1 bytes) 2024-09-13T09:18:33.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:33.824 INFO:teuthology.orchestra.run.smithi032.stdout:3751 write 0xd984600 thru 0xd988ad6 (0x44d7 bytes) 2024-09-13T09:18:33.831 INFO:teuthology.orchestra.run.smithi032.stdout:3752 punch from 0x22d4e02 to 0x22da85f, (0x5a5d bytes) 2024-09-13T09:18:33.831 INFO:teuthology.orchestra.run.smithi032.stdout:3758 write 0xb05c282 thru 0xb06b479 (0xf1f8 bytes) 2024-09-13T09:18:33.834 INFO:teuthology.orchestra.run.smithi032.stdout:3760 write 0xd06d2c9 thru 0xd070776 (0x34ae bytes) 2024-09-13T09:18:33.840 INFO:teuthology.orchestra.run.smithi032.stdout:3761 write 0x4104f58 thru 0x4114e2c (0xfed5 bytes) 2024-09-13T09:18:33.846 INFO:teuthology.orchestra.run.smithi032.stdout:3763 writesame 0x82a8f82 thru 0x82aec28 data_size 0x1(0x5ca7 bytes) 2024-09-13T09:18:33.854 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:33.854+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:33.882 INFO:teuthology.orchestra.run.smithi032.stdout:3765 clone 26 order 21 su 8192 sc 12 2024-09-13T09:18:33.941 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:33.941 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:33.941 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:33.941 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:33.941 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:33.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:33.956 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:33.950+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:34.057 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:34.050+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:34.128 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:34.128 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:34.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:34.128 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.589498679000002, 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.129 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:18:34.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:34.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:18:34.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:18:34.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:18:34.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:18:34.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:18:34.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:18:34.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:18:34.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:18:34.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:18:34.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32899 3.18 3:1fefe8db:::rbd_data.1185833655de.000000000000001e:16 [read 704512~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219951631000001, 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046124828, 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:34.135 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.136 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.136 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.136 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32899 2024-09-13T09:18:34.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.136 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:34.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:34.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905907+0000", 2024-09-13T09:18:34.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905909+0000", 2024-09-13T09:18:34.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:18:34.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905919+0000", 2024-09-13T09:18:34.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0520000000000007e-06 2024-09-13T09:18:34.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905923+0000", 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.951907+0000", 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.045983221999999997 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.951969+0000", 2024-09-13T09:18:34.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2442999999999998e-05 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952032+0000", 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999998e-05 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:34.140 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219931694, 2024-09-13T09:18:34.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:18:34.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.141 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:18:34.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.141 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:34.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:34.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:18:34.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:18:34.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:18:34.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:18:34.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:18:34.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:18:34.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:18:34.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.145 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.145 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219914373000002, 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:18:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:18:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:18:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:18:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:18:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:18:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219897432, 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:18:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:18:34.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:18:34.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:18:34.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:18:34.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:18:34.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:18:34.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:18:34.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:18:34.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:18:34.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:18:34.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:18:34.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.154 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.154 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.154 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:34.155 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219880203999999, 2024-09-13T09:18:34.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:18:34.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.155 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.155 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.155 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.155 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.155 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:18:34.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.155 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:34.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:34.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:34.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:18:34.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:18:34.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:18:34.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:18:34.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:18:34.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:18:34.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.159 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.159 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.159 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:34.159 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219843783000002, 2024-09-13T09:18:34.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:18:34.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.159 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:34.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:34.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:18:34.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:18:34.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:18:34.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:18:34.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:18:34.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:18:34.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.163 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219790321000001, 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:18:34.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:18:34.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:18:34.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:18:34.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:18:34.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:18:34.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:18:34.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:18:34.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:18:34.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219768300999998, 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.168 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.169 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.169 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:18:34.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.169 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:34.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:18:34.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:18:34.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:18:34.172 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.172 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.172 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.172 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.172 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:34.172 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219743665999999, 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:18:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:18:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:18:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:18:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219725143000002, 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:18:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:18:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:18:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:18:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:18:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:18:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:18:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:18:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:18:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:18:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219707272000001, 2024-09-13T09:18:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:18:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:18:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:18:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:18:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:18:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:18:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:18:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:18:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:18:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:18:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:18:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219689941999999, 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:18:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:18:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:18:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:18:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:18:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:18:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:18:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219672471999999, 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:18:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219655606, 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219637818999999, 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.219593508999999, 2024-09-13T09:18:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:18:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:18:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:18:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:18:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:18:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.986463020999999, 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:18:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:18:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:18:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.986446187999999, 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:18:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:18:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:18:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:18:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:18:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.986429829999999, 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:18:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:18:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:18:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:18:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:18:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:18:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:18:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:18:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:34.221 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:34.223 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:34.223 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:34.218+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:34.324 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:34.318+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:34.361 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:34.361 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:34.361 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:34.361 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:34.375 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:34.466 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:34.462+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:34.526 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:34.522+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:34.547 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:34.548 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:34.548 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:34.548 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:34.548 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:34.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:34.586 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-13T09:18:34.627 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:34.626+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:34.723 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:34.724 INFO:teuthology.orchestra.run.smithi032.stdout:3768 writesame 0xb9a0ee8 thru 0xb9ad92f data_size 0x4(0xca48 bytes) 2024-09-13T09:18:34.728 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:34.726+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:34.739 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:34.739 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:34.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:34.739 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:34.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4388 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.748410303999997, 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069349509000000004, 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4388 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985769+0000", 2024-09-13T09:18:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985775+0000", 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5230000000000001e-06 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985794+0000", 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8425999999999999e-05 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985809+0000", 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.486e-05 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.054965+0000", 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069156463000000001 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055036+0000", 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0780999999999999e-05 2024-09-13T09:18:34.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055120+0000", 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3486000000000007e-05 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4389 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.748359780999998, 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069473833999999998, 2024-09-13T09:18:34.744 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4389 2024-09-13T09:18:34.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.745 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:34.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985820+0000", 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985859+0000", 2024-09-13T09:18:34.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9279999999999997e-05 2024-09-13T09:18:34.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985881+0000", 2024-09-13T09:18:34.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2116999999999998e-05 2024-09-13T09:18:34.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985896+0000", 2024-09-13T09:18:34.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4698999999999999e-05 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055152+0000", 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069256568000000004 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055220+0000", 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8255000000000003e-05 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055294+0000", 2024-09-13T09:18:34.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3858999999999999e-05 2024-09-13T09:18:34.749 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.749 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.749 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.749 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4390 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.749 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:34.749 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.748264169000002, 2024-09-13T09:18:34.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069563635999999998, 2024-09-13T09:18:34.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4390 2024-09-13T09:18:34.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.750 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:34.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985915+0000", 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985922+0000", 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985941+0000", 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8260000000000001e-05 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985955+0000", 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4409999999999999e-05 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055325+0000", 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069370290000000001 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055395+0000", 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0158000000000002e-05 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055480+0000", 2024-09-13T09:18:34.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4340999999999998e-05 2024-09-13T09:18:34.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.754 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.754 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.754 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:34.754 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.748213102000001, 2024-09-13T09:18:34.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:18:34.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:18:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:18:34.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:18:34.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:18:34.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:18:34.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:18:34.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:18:34.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:18:34.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:18:34.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.759 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.759 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.759 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:34.759 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.748164365999997, 2024-09-13T09:18:34.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:18:34.759 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.759 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.759 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:18:34.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.760 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:34.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:34.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:18:34.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:18:34.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:18:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.748096697000001, 2024-09-13T09:18:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:18:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:18:34.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.765 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:34.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:18:34.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:18:34.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:18:34.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:18:34.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.768 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747956488, 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:34.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:18:34.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747865621999999, 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:34.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:18:34.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747790762000001, 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.778 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:18:34.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:34.782 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747719586000002, 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:34.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:18:34.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:18:34.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:18:34.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.787 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.787 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.787 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:34.787 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747670532999997, 2024-09-13T09:18:34.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:18:34.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:18:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:18:34.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747608796999998, 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:18:34.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.793 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:34.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:18:34.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747559127000002, 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:18:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:18:34.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:18:34.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:18:34.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:18:34.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:18:34.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.801 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747510607999999, 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747462212000002, 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:18:34.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:18:34.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747414925000001, 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.811 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:18:34.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747331523, 2024-09-13T09:18:34.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:18:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747257797000003, 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747189673999998, 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:18:34.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.747139207000004, 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:34.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:34.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:18:34.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:34.834 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:34.834 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:34.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:34.836 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:34.830+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:34.928 INFO:teuthology.orchestra.run.smithi032.stdout:3769 write 0xba7b095 thru 0xba808fd (0x5869 bytes) 2024-09-13T09:18:34.931 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:34.931 INFO:teuthology.orchestra.run.smithi032.stdout:3771 read 0x2cd6e1b thru 0x2cdd7fb (0x69e1 bytes) 2024-09-13T09:18:34.931 INFO:teuthology.orchestra.run.smithi032.stdout:3772 write 0x7647bdb thru 0x76518e0 (0x9d06 bytes) 2024-09-13T09:18:34.936 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:34.930+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:34.942 INFO:teuthology.orchestra.run.smithi032.stdout:3773 write 0x8ee8c0f thru 0x8ef75dd (0xe9cf bytes) 2024-09-13T09:18:35.012 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:35.012 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:35.012 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:35.012 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:35.028 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:35.069 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:35.066+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:35.095 INFO:teuthology.orchestra.run.smithi032.stdout:3775 read 0xb9b9871 thru 0xb9c2dcc (0x955c bytes) 2024-09-13T09:18:35.097 INFO:teuthology.orchestra.run.smithi032.stdout:3776 write 0xb2367ca thru 0xb2383a7 (0x1bde bytes) 2024-09-13T09:18:35.137 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:35.130+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:35.148 INFO:teuthology.orchestra.run.smithi032.stdout:3777 write 0x7fecadb thru 0x7ff8cf1 (0xc217 bytes) 2024-09-13T09:18:35.180 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:35.181 INFO:teuthology.orchestra.run.smithi032.stdout:3779 read 0x8b9ffdd thru 0x8badecf (0xdef3 bytes) 2024-09-13T09:18:35.182 INFO:teuthology.orchestra.run.smithi032.stdout:3782 read 0x134ddf7 thru 0x13584ce (0xa6d8 bytes) 2024-09-13T09:18:35.184 INFO:teuthology.orchestra.run.smithi032.stdout:3787 write 0x1a55e3f thru 0x1a5f2aa (0x946c bytes) 2024-09-13T09:18:35.200 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:35.201 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:35.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:18:35.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:35.258 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:35.254+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:35.339 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:35.338+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:35.343 INFO:teuthology.orchestra.run.smithi032.stdout:3788 write 0x145793e thru 0x14642fb (0xc9be bytes) 2024-09-13T09:18:35.401 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:18:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 66.711725224000006, 2024-09-13T09:18:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.21372158699999999, 2024-09-13T09:18:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685363+0000", 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685372+0000", 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4590000000000004e-06 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685394+0000", 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1616000000000001e-05 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685407+0000", 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3675e-05 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685484+0000", 2024-09-13T09:18:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7065999999999997e-05 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685493+0000", 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8070000000000006e-06 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690770+0000", 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052763869999999996 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690940+0000", 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017001900000000001 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.691238+0000", 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029794499999999998 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.692057+0000", 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081943000000000001 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:35.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.898933+0000", 2024-09-13T09:18:35.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.20687588200000001 2024-09-13T09:18:35.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:35.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899026+0000", 2024-09-13T09:18:35.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2777999999999998e-05 2024-09-13T09:18:35.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899086+0000", 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0188e-05 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:14590 3.1d 3:be60f898:::rbd_data.11853ecd5f5f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13672152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.855876784000003, 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052557382, 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 14590 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:35.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541207+0000", 2024-09-13T09:18:35.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-13T09:18:35.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562477+0000", 2024-09-13T09:18:35.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021270155999999998 2024-09-13T09:18:35.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562539+0000", 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1851000000000001e-05 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562565+0000", 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6194e-05 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562702+0000", 2024-09-13T09:18:35.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013731299999999999 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.563071+0000", 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036892899999999999 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.565167+0000", 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020961959999999998 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.585261+0000", 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020093443999999999 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593596+0000", 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083352849999999996 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593696+0000", 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100021 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593770+0000", 2024-09-13T09:18:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3889999999999999e-05 2024-09-13T09:18:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:14584 3.2 e39/22 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head v 39'64, mlcod=38'62)", 2024-09-13T09:18:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.853891138000002, 2024-09-13T09:18:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051546091000000002, 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543197+0000", 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588691+0000", 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045493475999999998 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588711+0000", 2024-09-13T09:18:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0485000000000001e-05 2024-09-13T09:18:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588721+0000", 2024-09-13T09:18:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.499e-06 2024-09-13T09:18:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588766+0000", 2024-09-13T09:18:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5228000000000001e-05 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588788+0000", 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2503000000000001e-05 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594671+0000", 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058822809999999996 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594744+0000", 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3776000000000007e-05 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:22580 3.1c 3:39876c9f:::rbd_data.1185d6f8ec87.000000000000000f:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.544638347999999, 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050372896, 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22580 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852449+0000", 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852462+0000", 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3756e-05 2024-09-13T09:18:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852495+0000", 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2435999999999999e-05 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852516+0000", 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1347000000000001e-05 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852644+0000", 2024-09-13T09:18:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012777200000000001 2024-09-13T09:18:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852784+0000", 2024-09-13T09:18:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013980500000000001 2024-09-13T09:18:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.860936+0000", 2024-09-13T09:18:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081526859999999993 2024-09-13T09:18:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902522+0000", 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041585203000000001 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902560+0000", 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8603000000000003e-05 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902708+0000", 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014740900000000001 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902824+0000", 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116519 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54746 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8011776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.16570787200000001, 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041426971999999999, 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54746 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231381+0000", 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231384+0000", 2024-09-13T09:18:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.599e-06 2024-09-13T09:18:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231390+0000", 2024-09-13T09:18:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:18:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231395+0000", 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0170000000000004e-06 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272716+0000", 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041321670999999997 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272767+0000", 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0447e-05 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272809+0000", 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1659999999999998e-05 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54986 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6348800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15479927299999999, 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041375935000000003, 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54986 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242292+0000", 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3039999999999999e-06 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242298+0000", 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242303+0000", 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8359999999999999e-06 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283539+0000", 2024-09-13T09:18:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041235330000000001 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283593+0000", 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4926000000000002e-05 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283666+0000", 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2718000000000004e-05 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54987 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6447104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15478209100000001, 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041604369000000002, 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54987 2024-09-13T09:18:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242309+0000", 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1160000000000002e-06 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242317+0000", 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9799999999999998e-06 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242323+0000", 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6439999999999997e-06 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283711+0000", 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388085999999998 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283766+0000", 2024-09-13T09:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5112999999999999e-05 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283912+0000", 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014575300000000001 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54988 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6545408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15476227100000001, 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041865404000000002, 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54988 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242329+0000", 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242335+0000", 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242341+0000", 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9440000000000002e-06 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283952+0000", 2024-09-13T09:18:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041610981999999998 2024-09-13T09:18:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284076+0000", 2024-09-13T09:18:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124356 2024-09-13T09:18:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284193+0000", 2024-09-13T09:18:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116345 2024-09-13T09:18:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54989 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6643712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15474415699999999, 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042123657000000002, 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54989 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242348+0000", 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.655e-06 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242354+0000", 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2269999999999998e-06 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242360+0000", 2024-09-13T09:18:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6170000000000003e-06 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284248+0000", 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041887830000000001 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284340+0000", 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1358000000000002e-05 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284469+0000", 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012932300000000001 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54990 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6742016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.154710867, 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042278333000000001, 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54990 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242378+0000", 2024-09-13T09:18:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242381+0000", 2024-09-13T09:18:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0680000000000001e-06 2024-09-13T09:18:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242392+0000", 2024-09-13T09:18:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0762e-05 2024-09-13T09:18:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242398+0000", 2024-09-13T09:18:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1040000000000003e-06 2024-09-13T09:18:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284506+0000", 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042107983000000002 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284564+0000", 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.821e-05 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284657+0000", 2024-09-13T09:18:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2588000000000002e-05 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54991 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6840320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15468607600000001, 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042489459, 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54991 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242406+0000", 2024-09-13T09:18:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:18:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242412+0000", 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5919999999999997e-06 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242425+0000", 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2713e-05 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284694+0000", 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042269558999999998 2024-09-13T09:18:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284784+0000", 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9999000000000003e-05 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284893+0000", 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108422 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54992 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6938624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15465970500000001, 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042636897999999999, 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54992 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242429+0000", 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242432+0000", 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7599999999999998e-06 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242439+0000", 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5760000000000002e-06 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242448+0000", 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284935+0000", 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042486949000000003 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284991+0000", 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6471000000000002e-05 2024-09-13T09:18:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285067+0000", 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5543000000000005e-05 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54993 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7036928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.154636828, 2024-09-13T09:18:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042867959999999997, 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54993 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242452+0000", 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242455+0000", 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4360000000000001e-06 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242463+0000", 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8220000000000007e-06 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242468+0000", 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9479999999999999e-06 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285107+0000", 2024-09-13T09:18:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042639415999999999 2024-09-13T09:18:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285196+0000", 2024-09-13T09:18:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8721000000000007e-05 2024-09-13T09:18:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285321+0000", 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012495 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54994 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7135232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15460966400000001, 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043009622999999997, 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54994 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242482+0000", 2024-09-13T09:18:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:18:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242486+0000", 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1049999999999997e-06 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242489+0000", 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.004e-06 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285354+0000", 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.04286479 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285413+0000", 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9778000000000003e-05 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285489+0000", 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6141000000000002e-05 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54995 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7233536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.154598293, 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043166641999999998, 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54995 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242492+0000", 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.437e-06 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242496+0000", 2024-09-13T09:18:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242498+0000", 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5950000000000001e-06 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285528+0000", 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043029755000000003 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285585+0000", 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285658+0000", 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2713999999999995e-05 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54996 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7331840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15458897799999999, 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043333782000000001, 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54996 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242500+0000", 2024-09-13T09:18:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242502+0000", 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3599999999999999e-06 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242505+0000", 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1439999999999999e-06 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242507+0000", 2024-09-13T09:18:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.605e-06 2024-09-13T09:18:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285698+0000", 2024-09-13T09:18:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043190911999999998 2024-09-13T09:18:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285754+0000", 2024-09-13T09:18:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5541999999999998e-05 2024-09-13T09:18:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285834+0000", 2024-09-13T09:18:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0426000000000006e-05 2024-09-13T09:18:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54997 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.154579889, 2024-09-13T09:18:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043496174999999998, 2024-09-13T09:18:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54997 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242509+0000", 2024-09-13T09:18:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242511+0000", 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.313e-06 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242514+0000", 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242516+0000", 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3999999999999999e-06 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285875+0000", 2024-09-13T09:18:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043358578000000002 2024-09-13T09:18:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285929+0000", 2024-09-13T09:18:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4602999999999999e-05 2024-09-13T09:18:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286006+0000", 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6429999999999998e-05 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54998 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7528448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15448727700000001, 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043666469999999999, 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54998 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242518+0000", 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999161 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242606+0000", 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7607000000000003e-05 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242617+0000", 2024-09-13T09:18:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0882e-05 2024-09-13T09:18:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242624+0000", 2024-09-13T09:18:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9009999999999997e-06 2024-09-13T09:18:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286046+0000", 2024-09-13T09:18:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043422308 2024-09-13T09:18:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286106+0000", 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0285000000000003e-05 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286269+0000", 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000162578 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:55005 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8216576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.245973+0000", 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15111691999999999, 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041347676, 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 55005 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245973+0000", 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245973+0000", 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245972+0000", 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245974+0000", 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2110000000000001e-06 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245982+0000", 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2219999999999996e-06 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245987+0000", 2024-09-13T09:18:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9139999999999999e-06 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287209+0000", 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222560999999998 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287265+0000", 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6081000000000001e-05 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287320+0000", 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4972999999999997e-05 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:55006 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8314880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15109860899999999, 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041461771000000001, 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 55006 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245993+0000", 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2869999999999999e-06 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245999+0000", 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0159999999999997e-06 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.246004+0000", 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287345+0000", 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041341217 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287398+0000", 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2840000000000002e-05 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287453+0000", 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4746000000000001e-05 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.503 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:35.505 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:35.505 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:35.502+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:35.545 INFO:teuthology.orchestra.run.smithi032.stdout:3789 write 0x7d9b103 thru 0x7d9db52 (0x2a50 bytes) 2024-09-13T09:18:35.547 INFO:teuthology.orchestra.run.smithi032.stdout:3790 write 0xa2b8db1 thru 0xa2b8f12 (0x162 bytes) 2024-09-13T09:18:35.551 INFO:teuthology.orchestra.run.smithi032.stdout:3791 write 0x1e1f754 thru 0x1e21529 (0x1dd6 bytes) 2024-09-13T09:18:35.598 INFO:teuthology.orchestra.run.smithi032.stdout:3793 trunc from 0xea9f773 to 0xde89f0 2024-09-13T09:18:35.607 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:35.602+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:35.670 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57598 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.631156+0000", 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012292456, 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012330172, 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57598 2024-09-13T09:18:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631156+0000", 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631156+0000", 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631155+0000", 2024-09-13T09:18:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631158+0000", 2024-09-13T09:18:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-13T09:18:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631167+0000", 2024-09-13T09:18:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1079999999999999e-06 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631174+0000", 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.443e-06 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57630 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632068+0000", 2024-09-13T09:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011380362, 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011526414, 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57630 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632068+0000", 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632068+0000", 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632067+0000", 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632070+0000", 2024-09-13T09:18:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1259999999999998e-06 2024-09-13T09:18:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632081+0000", 2024-09-13T09:18:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0386e-05 2024-09-13T09:18:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632087+0000", 2024-09-13T09:18:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5880000000000003e-06 2024-09-13T09:18:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57832 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633017+0000", 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010430762, 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010675943, 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57832 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633017+0000", 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633017+0000", 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633017+0000", 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633020+0000", 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633027+0000", 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5340000000000002e-06 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633034+0000", 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3489999999999997e-06 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57864 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633994+0000", 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0094538999999999995, 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.009858861, 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57864 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633994+0000", 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633994+0000", 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633993+0000", 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633997+0000", 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0879999999999999e-06 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634004+0000", 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7829999999999999e-06 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634011+0000", 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5139999999999998e-06 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57896 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634977+0000", 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.008470857, 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089669369999999995, 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57896 2024-09-13T09:18:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634977+0000", 2024-09-13T09:18:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634977+0000", 2024-09-13T09:18:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634977+0000", 2024-09-13T09:18:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634984+0000", 2024-09-13T09:18:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4630000000000004e-06 2024-09-13T09:18:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634997+0000", 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2857000000000001e-05 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635004+0000", 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58098 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635763+0000", 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0076846559999999998, 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082608579999999994, 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58098 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635763+0000", 2024-09-13T09:18:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635763+0000", 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635763+0000", 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635766+0000", 2024-09-13T09:18:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:18:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635773+0000", 2024-09-13T09:18:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.469e-06 2024-09-13T09:18:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635780+0000", 2024-09-13T09:18:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4679999999999997e-06 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58130 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636551+0000", 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0068971609999999997, 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075521759999999999, 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58130 2024-09-13T09:18:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636551+0000", 2024-09-13T09:18:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636551+0000", 2024-09-13T09:18:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636550+0000", 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636553+0000", 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636564+0000", 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0531e-05 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636570+0000", 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58332 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637323+0000", 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0061249110000000002, 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068963940000000001, 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58332 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637323+0000", 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637323+0000", 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637323+0000", 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637326+0000", 2024-09-13T09:18:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0479999999999999e-06 2024-09-13T09:18:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637334+0000", 2024-09-13T09:18:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.011e-06 2024-09-13T09:18:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637340+0000", 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4180000000000002e-06 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57599 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.631179+0000", 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012268970000000001, 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013162023, 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57599 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631179+0000", 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631179+0000", 2024-09-13T09:18:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631179+0000", 2024-09-13T09:18:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631182+0000", 2024-09-13T09:18:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1140000000000002e-06 2024-09-13T09:18:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631189+0000", 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7600000000000002e-06 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631196+0000", 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.657e-06 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57631 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632092+0000", 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011355687999999999, 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012331923, 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57631 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632092+0000", 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632092+0000", 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632092+0000", 2024-09-13T09:18:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632095+0000", 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632103+0000", 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8769999999999993e-06 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632111+0000", 2024-09-13T09:18:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9310000000000008e-06 2024-09-13T09:18:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57833 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633039+0000", 2024-09-13T09:18:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010409382, 2024-09-13T09:18:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011462328000000001, 2024-09-13T09:18:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57833 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633039+0000", 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633039+0000", 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633038+0000", 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633041+0000", 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8250000000000001e-06 2024-09-13T09:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633049+0000", 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633055+0000", 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.263e-06 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57865 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634016+0000", 2024-09-13T09:18:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0094320089999999999, 2024-09-13T09:18:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010561189, 2024-09-13T09:18:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57865 2024-09-13T09:18:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634016+0000", 2024-09-13T09:18:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634016+0000", 2024-09-13T09:18:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634016+0000", 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634018+0000", 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8260000000000001e-06 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634026+0000", 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-13T09:18:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634033+0000", 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6630000000000004e-06 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57897 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635009+0000", 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.008439106, 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096792449999999995, 2024-09-13T09:18:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57897 2024-09-13T09:18:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635009+0000", 2024-09-13T09:18:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635009+0000", 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635008+0000", 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635011+0000", 2024-09-13T09:18:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.932e-06 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635019+0000", 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7570000000000004e-06 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635026+0000", 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4049999999999997e-06 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58099 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635785+0000", 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0076630709999999996, 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090148659999999999, 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58099 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635785+0000", 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635785+0000", 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635784+0000", 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635787+0000", 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8109999999999999e-06 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635795+0000", 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.435e-06 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635801+0000", 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3169999999999998e-06 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58131 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636575+0000", 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0068726209999999998, 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083046779999999994, 2024-09-13T09:18:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58131 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636575+0000", 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636575+0000", 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636575+0000", 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636578+0000", 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8430000000000001e-06 2024-09-13T09:18:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636586+0000", 2024-09-13T09:18:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8830000000000006e-06 2024-09-13T09:18:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636592+0000", 2024-09-13T09:18:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4139999999999999e-06 2024-09-13T09:18:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58333 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637357+0000", 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0060915509999999997, 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.007602218, 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58333 2024-09-13T09:18:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637357+0000", 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637357+0000", 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637356+0000", 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637364+0000", 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0600000000000008e-06 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637373+0000", 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0080000000000008e-06 2024-09-13T09:18:35.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637380+0000", 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6540000000000002e-06 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57600 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.631202+0000", 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012246251, 2024-09-13T09:18:35.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013833280999999999, 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57600 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631202+0000", 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631202+0000", 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631201+0000", 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631204+0000", 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0860000000000002e-06 2024-09-13T09:18:35.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631212+0000", 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7079999999999997e-06 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631220+0000", 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7579999999999992e-06 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57632 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632116+0000", 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011332276000000001, 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013030158999999999, 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57632 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632116+0000", 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632116+0000", 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632115+0000", 2024-09-13T09:18:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632118+0000", 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632126+0000", 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632132+0000", 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4849999999999997e-06 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57834 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633063+0000", 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010384671, 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012200196, 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.736 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57834 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633063+0000", 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633063+0000", 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633063+0000", 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633066+0000", 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0570000000000001e-06 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633073+0000", 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633080+0000", 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.740 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57866 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.740 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634038+0000", 2024-09-13T09:18:35.740 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0094101059999999997, 2024-09-13T09:18:35.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011303809999999999, 2024-09-13T09:18:35.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.740 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57866 2024-09-13T09:18:35.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634038+0000", 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634038+0000", 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634037+0000", 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634040+0000", 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8679999999999999e-06 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634048+0000", 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5830000000000001e-06 2024-09-13T09:18:35.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634054+0000", 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4400000000000002e-06 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57898 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635031+0000", 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.008417318, 2024-09-13T09:18:35.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010388272, 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57898 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635031+0000", 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635031+0000", 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635030+0000", 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635033+0000", 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8679999999999999e-06 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635041+0000", 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6059999999999997e-06 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635047+0000", 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4080000000000003e-06 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58100 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635807+0000", 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.007641531, 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096881959999999996, 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58100 2024-09-13T09:18:35.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635807+0000", 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635807+0000", 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635806+0000", 2024-09-13T09:18:35.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635809+0000", 2024-09-13T09:18:35.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-13T09:18:35.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635816+0000", 2024-09-13T09:18:35.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6079999999999998e-06 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635823+0000", 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3119999999999999e-06 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58132 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636597+0000", 2024-09-13T09:18:35.750 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0068507810000000002, 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090079849999999996, 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58132 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636597+0000", 2024-09-13T09:18:35.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636597+0000", 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636597+0000", 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636600+0000", 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.926e-06 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636607+0000", 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5510000000000002e-06 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636615+0000", 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6369999999999999e-06 2024-09-13T09:18:35.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.754 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.754 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58334 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.754 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637386+0000", 2024-09-13T09:18:35.754 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0060626029999999997, 2024-09-13T09:18:35.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083309920000000006, 2024-09-13T09:18:35.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58334 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637386+0000", 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637386+0000", 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637385+0000", 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637388+0000", 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.038e-06 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637396+0000", 2024-09-13T09:18:35.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7190000000000001e-06 2024-09-13T09:18:35.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637402+0000", 2024-09-13T09:18:35.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.195e-06 2024-09-13T09:18:35.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.757 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.757 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57601 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.631228+0000", 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012220111000000001, 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014566951, 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57601 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631228+0000", 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631228+0000", 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631227+0000", 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631230+0000", 2024-09-13T09:18:35.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.92e-06 2024-09-13T09:18:35.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631238+0000", 2024-09-13T09:18:35.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7999999999999999e-06 2024-09-13T09:18:35.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.631245+0000", 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3720000000000001e-06 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57633 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632138+0000", 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011310302, 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013760237999999999, 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.761 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57633 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632138+0000", 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632138+0000", 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632137+0000", 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632140+0000", 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632148+0000", 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6229999999999997e-06 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632154+0000", 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2090000000000002e-06 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.765 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57835 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.765 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633085+0000", 2024-09-13T09:18:35.765 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010363294, 2024-09-13T09:18:35.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012897529, 2024-09-13T09:18:35.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57835 2024-09-13T09:18:35.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.765 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633085+0000", 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633085+0000", 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633084+0000", 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633103+0000", 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8257000000000002e-05 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633112+0000", 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4399999999999994e-06 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633119+0000", 2024-09-13T09:18:35.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7460000000000003e-06 2024-09-13T09:18:35.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.768 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.768 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57867 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.768 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634063+0000", 2024-09-13T09:18:35.768 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0093855580000000004, 2024-09-13T09:18:35.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012076406, 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57867 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634063+0000", 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634063+0000", 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634062+0000", 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634065+0000", 2024-09-13T09:18:35.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0079999999999998e-06 2024-09-13T09:18:35.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634073+0000", 2024-09-13T09:18:35.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7489999999999998e-06 2024-09-13T09:18:35.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634079+0000", 2024-09-13T09:18:35.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1619999999999996e-06 2024-09-13T09:18:35.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.771 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57899 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635055+0000", 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0083932009999999994, 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011243927000000001, 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.772 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57899 2024-09-13T09:18:35.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.773 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635055+0000", 2024-09-13T09:18:35.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635055+0000", 2024-09-13T09:18:35.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635054+0000", 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635057+0000", 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0230000000000001e-06 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635065+0000", 2024-09-13T09:18:35.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8599999999999993e-06 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635071+0000", 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2400000000000004e-06 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58101 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635828+0000", 2024-09-13T09:18:35.775 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0076199170000000004, 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010634996000000001, 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58101 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635828+0000", 2024-09-13T09:18:35.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635828+0000", 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635828+0000", 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635830+0000", 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8059999999999999e-06 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635838+0000", 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7049999999999999e-06 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635844+0000", 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58133 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636620+0000", 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0068281180000000002, 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010003369999999999, 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58133 2024-09-13T09:18:35.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636620+0000", 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636620+0000", 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636619+0000", 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636622+0000", 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636630+0000", 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6629999999999993e-06 2024-09-13T09:18:35.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636638+0000", 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5050000000000001e-06 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58335 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637407+0000", 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.006041177, 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093773190000000003, 2024-09-13T09:18:35.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58335 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637407+0000", 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637407+0000", 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637406+0000", 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637409+0000", 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:18:35.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637417+0000", 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5490000000000001e-06 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637423+0000", 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4200000000000004e-06 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57634 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632161+0000", 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011287604999999999, 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014730994000000001, 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57634 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632161+0000", 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632161+0000", 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632160+0000", 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632163+0000", 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.717e-06 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632170+0000", 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7340000000000008e-06 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632177+0000", 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4200000000000004e-06 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57836 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633125+0000", 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010323354, 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013884168000000001, 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57836 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633125+0000", 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633125+0000", 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633124+0000", 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633127+0000", 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.117e-06 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633135+0000", 2024-09-13T09:18:35.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8169999999999999e-06 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633141+0000", 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57868 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634084+0000", 2024-09-13T09:18:35.792 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0093641809999999992, 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013007400000000001, 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57868 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634084+0000", 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634084+0000", 2024-09-13T09:18:35.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634083+0000", 2024-09-13T09:18:35.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634086+0000", 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8739999999999999e-06 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634094+0000", 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6680000000000001e-06 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634100+0000", 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3289999999999999e-06 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.795 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57900 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635089+0000", 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0083590069999999999, 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012171924000000001, 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.796 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57900 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635089+0000", 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635089+0000", 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635088+0000", 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635092+0000", 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1480000000000002e-06 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635100+0000", 2024-09-13T09:18:35.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5350000000000004e-06 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635109+0000", 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2189999999999993e-06 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58102 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635850+0000", 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0075985000000000002, 2024-09-13T09:18:35.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011495248, 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58102 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635850+0000", 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635850+0000", 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635849+0000", 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635868+0000", 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239e-05 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635879+0000", 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.118e-05 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635887+0000", 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.075e-06 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58134 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636643+0000", 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0068053389999999997, 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010788417999999999, 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58134 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.803 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636643+0000", 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636643+0000", 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636642+0000", 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636645+0000", 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.791e-06 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636653+0000", 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7999999999999999e-06 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636659+0000", 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58336 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637924+0000", 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0055236790000000001, 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095827629999999994, 2024-09-13T09:18:35.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58336 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637924+0000", 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637924+0000", 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637428+0000", 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9995036 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637932+0000", 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00050362900000000001 2024-09-13T09:18:35.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637946+0000", 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4572e-05 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637959+0000", 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2788e-05 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.810 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57635 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.810 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632182+0000", 2024-09-13T09:18:35.810 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011265819999999999, 2024-09-13T09:18:35.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015423427999999999, 2024-09-13T09:18:35.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57635 2024-09-13T09:18:35.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632182+0000", 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632182+0000", 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632182+0000", 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632185+0000", 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9399999999999998e-06 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632192+0000", 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7139999999999993e-06 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632199+0000", 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5939999999999999e-06 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57837 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633147+0000", 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010301512000000001, 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014569811, 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.813 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57837 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633147+0000", 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633147+0000", 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633146+0000", 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633149+0000", 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633157+0000", 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7680000000000008e-06 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633163+0000", 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5259999999999999e-06 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.816 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57869 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634136+0000", 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0093116929999999994, 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013657648, 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57869 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634136+0000", 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634136+0000", 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634136+0000", 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634139+0000", 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3510000000000002e-06 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634148+0000", 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5769999999999994e-06 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634154+0000", 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7950000000000002e-06 2024-09-13T09:18:35.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57901 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635115+0000", 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0083331159999999998, 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012762145000000001, 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.820 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57901 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635115+0000", 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635115+0000", 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635114+0000", 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635117+0000", 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.994e-06 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635125+0000", 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7970000000000001e-06 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635132+0000", 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5919999999999997e-06 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58103 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635892+0000", 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.007555717, 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012064745, 2024-09-13T09:18:35.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58103 2024-09-13T09:18:35.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.824 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635892+0000", 2024-09-13T09:18:35.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635892+0000", 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635892+0000", 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635895+0000", 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0029999999999999e-06 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635902+0000", 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6680000000000001e-06 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635909+0000", 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4320000000000004e-06 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.826 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58135 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636664+0000", 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0067839129999999999, 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01137296, 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58135 2024-09-13T09:18:35.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.828 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636664+0000", 2024-09-13T09:18:35.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636664+0000", 2024-09-13T09:18:35.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636664+0000", 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636666+0000", 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8150000000000002e-06 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636674+0000", 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.525e-06 2024-09-13T09:18:35.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636680+0000", 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58337 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637969+0000", 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0054795340000000003, 2024-09-13T09:18:35.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010144835, 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58337 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637969+0000", 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637969+0000", 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637968+0000", 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637971+0000", 2024-09-13T09:18:35.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3799999999999998e-06 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637979+0000", 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9279999999999993e-06 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637986+0000", 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9e-06 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57636 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632204+0000", 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011244094, 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01600656, 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57636 2024-09-13T09:18:35.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632204+0000", 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632204+0000", 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632203+0000", 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632206+0000", 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8030000000000001e-06 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632214+0000", 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0080000000000002e-06 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632221+0000", 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57838 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633172+0000", 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010276505999999999, 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015153959, 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57838 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633172+0000", 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633172+0000", 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633171+0000", 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633174+0000", 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9569999999999998e-06 2024-09-13T09:18:35.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633182+0000", 2024-09-13T09:18:35.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9030000000000004e-06 2024-09-13T09:18:35.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633188+0000", 2024-09-13T09:18:35.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3969999999999999e-06 2024-09-13T09:18:35.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.840 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.841 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57870 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.841 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634160+0000", 2024-09-13T09:18:35.841 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0092880410000000004, 2024-09-13T09:18:35.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014250973, 2024-09-13T09:18:35.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.841 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57870 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634160+0000", 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634160+0000", 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634159+0000", 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634162+0000", 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8339999999999999e-06 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634170+0000", 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.763e-06 2024-09-13T09:18:35.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634177+0000", 2024-09-13T09:18:35.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4679999999999997e-06 2024-09-13T09:18:35.844 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.844 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.844 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.844 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58072 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.844 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635137+0000", 2024-09-13T09:18:35.844 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0083110500000000004, 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013350402000000001, 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58072 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635137+0000", 2024-09-13T09:18:35.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635137+0000", 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635137+0000", 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635140+0000", 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1149999999999998e-06 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635147+0000", 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7800000000000001e-06 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635154+0000", 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3450000000000002e-06 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.847 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.848 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.848 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58104 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.848 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635914+0000", 2024-09-13T09:18:35.848 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0075340659999999999, 2024-09-13T09:18:35.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012655563, 2024-09-13T09:18:35.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58104 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635914+0000", 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635914+0000", 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635913+0000", 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635916+0000", 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8540000000000001e-06 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635927+0000", 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0488e-05 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635934+0000", 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7920000000000004e-06 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58136 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636686+0000", 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0067624759999999999, 2024-09-13T09:18:35.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011960178, 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58136 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636686+0000", 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636686+0000", 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636685+0000", 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636688+0000", 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8629999999999999e-06 2024-09-13T09:18:35.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636699+0000", 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1345999999999999e-05 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636706+0000", 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6599999999999998e-06 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.855 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58338 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.855 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.641911+0000", 2024-09-13T09:18:35.855 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0015373819999999999, 2024-09-13T09:18:35.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068147800000000003, 2024-09-13T09:18:35.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58338 2024-09-13T09:18:35.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.856 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.641911+0000", 2024-09-13T09:18:35.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.641911+0000", 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637991+0000", 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9960799 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.641918+0000", 2024-09-13T09:18:35.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039266359999999998 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.641936+0000", 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8467000000000001e-05 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.641952+0000", 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6013000000000001e-05 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57637 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632226+0000", 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0112224, 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016592583000000001, 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57637 2024-09-13T09:18:35.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632226+0000", 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632226+0000", 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632225+0000", 2024-09-13T09:18:35.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632228+0000", 2024-09-13T09:18:35.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.931e-06 2024-09-13T09:18:35.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632236+0000", 2024-09-13T09:18:35.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6660000000000008e-06 2024-09-13T09:18:35.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632242+0000", 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4829999999999996e-06 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57839 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633194+0000", 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010254561000000001, 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015733817000000001, 2024-09-13T09:18:35.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57839 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633194+0000", 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633194+0000", 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633193+0000", 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633196+0000", 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633204+0000", 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6879999999999999e-06 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633211+0000", 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6369999999999999e-06 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.865 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57871 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634201+0000", 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0092475419999999992, 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014813029, 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57871 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634201+0000", 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634201+0000", 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634200+0000", 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634203+0000", 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3380000000000001e-06 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634212+0000", 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5189999999999992e-06 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634218+0000", 2024-09-13T09:18:35.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:18:35.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.869 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.869 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58073 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.869 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635159+0000", 2024-09-13T09:18:35.869 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0082890680000000001, 2024-09-13T09:18:35.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013930763000000001, 2024-09-13T09:18:35.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58073 2024-09-13T09:18:35.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.870 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635159+0000", 2024-09-13T09:18:35.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635159+0000", 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635158+0000", 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635161+0000", 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7999999999999999e-06 2024-09-13T09:18:35.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635169+0000", 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6690000000000006e-06 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635175+0000", 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1940000000000003e-06 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58105 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635939+0000", 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0075089919999999999, 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013234292, 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58105 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635939+0000", 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635939+0000", 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635939+0000", 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635941+0000", 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7599999999999998e-06 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635949+0000", 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5950000000000001e-06 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635956+0000", 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5909999999999998e-06 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.875 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58137 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636713+0000", 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0067354620000000002, 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012536815, 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58137 2024-09-13T09:18:35.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636713+0000", 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636713+0000", 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636712+0000", 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636715+0000", 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8710000000000001e-06 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636723+0000", 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7980000000000006e-06 2024-09-13T09:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636732+0000", 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58339 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.641973+0000", 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0014754029999999999, 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073567270000000004, 2024-09-13T09:18:35.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58339 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.641973+0000", 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.641973+0000", 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.641972+0000", 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.641976+0000", 2024-09-13T09:18:35.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1470000000000003e-06 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.641986+0000", 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6129999999999993e-06 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.641996+0000", 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0597e-05 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.882 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57638 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632271+0000", 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011176858, 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017151141000000002, 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57638 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632271+0000", 2024-09-13T09:18:35.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632271+0000", 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632270+0000", 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632274+0000", 2024-09-13T09:18:35.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2349999999999999e-06 2024-09-13T09:18:35.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632282+0000", 2024-09-13T09:18:35.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6540000000000005e-06 2024-09-13T09:18:35.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632289+0000", 2024-09-13T09:18:35.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4309999999999999e-06 2024-09-13T09:18:35.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57840 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633221+0000", 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010227320999999999, 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016312951999999999, 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57840 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633221+0000", 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633221+0000", 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633220+0000", 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633223+0000", 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0920000000000002e-06 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633235+0000", 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2245e-05 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633242+0000", 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1260000000000003e-06 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57872 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634256+0000", 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0091924710000000007, 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015356521, 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.889 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57872 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634256+0000", 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634256+0000", 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634255+0000", 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634264+0000", 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634273+0000", 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1339999999999993e-06 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634280+0000", 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7800000000000003e-06 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58074 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635181+0000", 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0082671589999999996, 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014512188000000001, 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58074 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635181+0000", 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635181+0000", 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635180+0000", 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635183+0000", 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7429999999999998e-06 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635191+0000", 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635197+0000", 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58106 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635962+0000", 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0074861119999999996, 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013807138, 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58106 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635962+0000", 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635962+0000", 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635961+0000", 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635964+0000", 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635972+0000", 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5920000000000003e-06 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635978+0000", 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.331e-06 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.899 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58138 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636737+0000", 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0067110169999999997, 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013142249999999999, 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58138 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636737+0000", 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636737+0000", 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636737+0000", 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636739+0000", 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6910000000000002e-06 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636747+0000", 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636753+0000", 2024-09-13T09:18:35.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4570000000000002e-06 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58340 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.642002+0000", 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.001445703, 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079561979999999994, 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58340 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642002+0000", 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642002+0000", 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642002+0000", 2024-09-13T09:18:35.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642005+0000", 2024-09-13T09:18:35.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.507e-06 2024-09-13T09:18:35.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642014+0000", 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.49e-06 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642021+0000", 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9589999999999998e-06 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.906 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57639 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632297+0000", 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011150706999999999, 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017754536000000001, 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57639 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632297+0000", 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632297+0000", 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632297+0000", 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632300+0000", 2024-09-13T09:18:35.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9280000000000002e-06 2024-09-13T09:18:35.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632308+0000", 2024-09-13T09:18:35.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9000000000000006e-06 2024-09-13T09:18:35.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632314+0000", 2024-09-13T09:18:35.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1970000000000001e-06 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57841 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633258+0000", 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010189993, 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016922454999999999, 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57841 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633258+0000", 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633258+0000", 2024-09-13T09:18:35.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633258+0000", 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633261+0000", 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633269+0000", 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5399999999999996e-06 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633276+0000", 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3860000000000004e-06 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57873 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.913 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634306+0000", 2024-09-13T09:18:35.914 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0091417229999999992, 2024-09-13T09:18:35.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015952904, 2024-09-13T09:18:35.914 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.914 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.914 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.914 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.914 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57873 2024-09-13T09:18:35.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.914 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634306+0000", 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634306+0000", 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634306+0000", 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634309+0000", 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6660000000000001e-06 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634318+0000", 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8340000000000005e-06 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634325+0000", 2024-09-13T09:18:35.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4309999999999999e-06 2024-09-13T09:18:35.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.917 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.917 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58075 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.917 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635203+0000", 2024-09-13T09:18:35.917 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0082456049999999996, 2024-09-13T09:18:35.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015281863999999999, 2024-09-13T09:18:35.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.917 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58075 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635203+0000", 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635203+0000", 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635202+0000", 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635205+0000", 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8660000000000002e-06 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635212+0000", 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5800000000000003e-06 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635219+0000", 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1419999999999998e-06 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.920 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58107 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636022+0000", 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0074259160000000003, 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014540091, 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58107 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636022+0000", 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636022+0000", 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636021+0000", 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636026+0000", 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2540000000000003e-06 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636034+0000", 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7600000000000008e-06 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636041+0000", 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2029999999999997e-06 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58139 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636776+0000", 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0066723140000000004, 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013867345999999999, 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.924 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.925 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.925 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.925 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58139 2024-09-13T09:18:35.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.925 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636776+0000", 2024-09-13T09:18:35.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636776+0000", 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636775+0000", 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636783+0000", 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0399999999999993e-06 2024-09-13T09:18:35.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636793+0000", 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5340000000000005e-06 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636799+0000", 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6059999999999999e-06 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58341 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.642029+0000", 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.001418767, 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086920690000000002, 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58341 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642029+0000", 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642029+0000", 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642029+0000", 2024-09-13T09:18:35.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642032+0000", 2024-09-13T09:18:35.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1130000000000001e-06 2024-09-13T09:18:35.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642042+0000", 2024-09-13T09:18:35.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9029999999999999e-06 2024-09-13T09:18:35.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642049+0000", 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9800000000000001e-06 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57640 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632322+0000", 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011125618, 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018497601999999998, 2024-09-13T09:18:35.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57640 2024-09-13T09:18:35.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.932 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632322+0000", 2024-09-13T09:18:35.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632322+0000", 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632322+0000", 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632325+0000", 2024-09-13T09:18:35.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632333+0000", 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7649999999999993e-06 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632339+0000", 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2779999999999999e-06 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57842 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633281+0000", 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010166945, 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017652217000000001, 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57842 2024-09-13T09:18:35.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633281+0000", 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633281+0000", 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633280+0000", 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633284+0000", 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0769999999999999e-06 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633291+0000", 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7719999999999994e-06 2024-09-13T09:18:35.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633297+0000", 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1630000000000001e-06 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57874 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634330+0000", 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0091180800000000006, 2024-09-13T09:18:35.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01668704, 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57874 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634330+0000", 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634330+0000", 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634329+0000", 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634332+0000", 2024-09-13T09:18:35.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.892e-06 2024-09-13T09:18:35.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634340+0000", 2024-09-13T09:18:35.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7200000000000006e-06 2024-09-13T09:18:35.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634347+0000", 2024-09-13T09:18:35.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58076 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635224+0000", 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0082243879999999991, 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015869561000000001, 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.942 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58076 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635224+0000", 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635224+0000", 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635223+0000", 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635226+0000", 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6970000000000002e-06 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635233+0000", 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.588e-06 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635240+0000", 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4289999999999998e-06 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58108 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.945 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636046+0000", 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0074022100000000002, 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015127448, 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58108 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636046+0000", 2024-09-13T09:18:35.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636046+0000", 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636045+0000", 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636048+0000", 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9249999999999999e-06 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636056+0000", 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5950000000000001e-06 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636062+0000", 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:18:35.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58140 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636805+0000", 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0066433630000000002, 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014444780000000001, 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58140 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636805+0000", 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636805+0000", 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636804+0000", 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636807+0000", 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9000000000000002e-06 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636815+0000", 2024-09-13T09:18:35.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8140000000000001e-06 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636821+0000", 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3489999999999997e-06 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58342 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.642055+0000", 2024-09-13T09:18:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0013935040000000001, 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092749349999999998, 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58342 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642055+0000", 2024-09-13T09:18:35.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642055+0000", 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642054+0000", 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642057+0000", 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1829999999999998e-06 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642066+0000", 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3440000000000001e-06 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642073+0000", 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1459999999999999e-06 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.955 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57641 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632344+0000", 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011104067, 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019078957000000001, 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57641 2024-09-13T09:18:35.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632344+0000", 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632344+0000", 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632343+0000", 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632351+0000", 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2139999999999999e-06 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632359+0000", 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.926e-06 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632365+0000", 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3509999999999998e-06 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57843 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633322+0000", 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010126415999999999, 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018216441999999999, 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57843 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633322+0000", 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633322+0000", 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633321+0000", 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633325+0000", 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9110000000000003e-06 2024-09-13T09:18:35.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633334+0000", 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633340+0000", 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.46e-06 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.963 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57875 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.963 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634352+0000", 2024-09-13T09:18:35.963 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0090959890000000005, 2024-09-13T09:18:35.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017270031000000002, 2024-09-13T09:18:35.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57875 2024-09-13T09:18:35.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.963 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634352+0000", 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634352+0000", 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634351+0000", 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634354+0000", 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.095e-06 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634362+0000", 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7400000000000004e-06 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634368+0000", 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1480000000000003e-06 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58077 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635245+0000", 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0082032400000000005, 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016453224999999998, 2024-09-13T09:18:35.966 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58077 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635245+0000", 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635245+0000", 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635244+0000", 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635247+0000", 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7250000000000002e-06 2024-09-13T09:18:35.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635255+0000", 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5490000000000001e-06 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635261+0000", 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1539999999999999e-06 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58109 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636067+0000", 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0073806820000000004, 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015714174000000001, 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58109 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.970 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636067+0000", 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636067+0000", 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636067+0000", 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636070+0000", 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.774e-06 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636086+0000", 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6697e-05 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636094+0000", 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8890000000000002e-06 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58141 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636826+0000", 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0066216570000000004, 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015060917, 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.973 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58141 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636826+0000", 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636826+0000", 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636826+0000", 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636829+0000", 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7580000000000001e-06 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636836+0000", 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6969999999999993e-06 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636843+0000", 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4649999999999999e-06 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.976 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58343 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.642079+0000", 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0013695739999999999, 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099471980000000008, 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58343 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642079+0000", 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642079+0000", 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642078+0000", 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642081+0000", 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1300000000000001e-06 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642108+0000", 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7339999999999999e-05 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642115+0000", 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6710000000000002e-06 2024-09-13T09:18:35.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57642 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632385+0000", 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011063359, 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019738740000000001, 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57642 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632385+0000", 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632385+0000", 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632384+0000", 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632387+0000", 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3569999999999998e-06 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632396+0000", 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6519999999999995e-06 2024-09-13T09:18:35.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632403+0000", 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.545e-06 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57844 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633346+0000", 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010102171, 2024-09-13T09:18:35.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018905142, 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57844 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633346+0000", 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633346+0000", 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633345+0000", 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633348+0000", 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.106e-06 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633356+0000", 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9969999999999998e-06 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633362+0000", 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1909999999999997e-06 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.986 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57876 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634374+0000", 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0090743490000000007, 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017957770000000001, 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57876 2024-09-13T09:18:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634374+0000", 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634374+0000", 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634373+0000", 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634376+0000", 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8890000000000002e-06 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634384+0000", 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6399999999999997e-06 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634390+0000", 2024-09-13T09:18:35.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4200000000000004e-06 2024-09-13T09:18:35.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.990 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.990 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58078 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.990 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635266+0000", 2024-09-13T09:18:35.990 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0081822660000000005, 2024-09-13T09:18:35.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017146721, 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58078 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635266+0000", 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635266+0000", 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635265+0000", 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635272+0000", 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635280+0000", 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0190000000000006e-06 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635287+0000", 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3090000000000001e-06 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58110 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636100+0000", 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0073482310000000002, 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016390040000000002, 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58110 2024-09-13T09:18:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636100+0000", 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636100+0000", 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636099+0000", 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636102+0000", 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.914e-06 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636110+0000", 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1000000000000004e-06 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636118+0000", 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4059999999999999e-06 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58142 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636848+0000", 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0065999429999999996, 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015771212999999999, 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:35.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58142 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636848+0000", 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:35.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636848+0000", 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636848+0000", 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636850+0000", 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9249999999999999e-06 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636858+0000", 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7519999999999996e-06 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636865+0000", 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3199999999999996e-06 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58344 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.642122+0000", 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0013261530000000001, 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010609794000000001, 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58344 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642122+0000", 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642122+0000", 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642121+0000", 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642124+0000", 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.157e-06 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642137+0000", 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2211999999999999e-05 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642143+0000", 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6880000000000002e-06 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57643 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632442+0000", 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.011006319000000001, 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020383976000000002, 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57643 2024-09-13T09:18:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632442+0000", 2024-09-13T09:18:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632442+0000", 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632441+0000", 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632445+0000", 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0060000000000003e-06 2024-09-13T09:18:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632454+0000", 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6079999999999996e-06 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632460+0000", 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5459999999999997e-06 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57845 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633368+0000", 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010080234, 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019577592000000001, 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57845 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633368+0000", 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633368+0000", 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633367+0000", 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633371+0000", 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.585e-06 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633385+0000", 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4039999999999999e-05 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633396+0000", 2024-09-13T09:18:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0637e-05 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57877 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634395+0000", 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0090527439999999997, 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018628255999999999, 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57877 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634395+0000", 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634395+0000", 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634395+0000", 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634398+0000", 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.943e-06 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634405+0000", 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6629999999999993e-06 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634412+0000", 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2310000000000001e-06 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58079 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635294+0000", 2024-09-13T09:18:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0081541659999999992, 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017831016000000002, 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58079 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635294+0000", 2024-09-13T09:18:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635294+0000", 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635293+0000", 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635296+0000", 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.943e-06 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635304+0000", 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0399999999999993e-06 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635311+0000", 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1659999999999999e-06 2024-09-13T09:18:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58111 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636124+0000", 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0073244310000000002, 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017077928999999999, 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58111 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636124+0000", 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636124+0000", 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636123+0000", 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636126+0000", 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8619999999999999e-06 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636134+0000", 2024-09-13T09:18:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7719999999999994e-06 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636140+0000", 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3080000000000004e-06 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58143 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636870+0000", 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0065778549999999996, 2024-09-13T09:18:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016415583000000001, 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58143 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636870+0000", 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636870+0000", 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636870+0000", 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636872+0000", 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5399999999999998e-06 2024-09-13T09:18:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636880+0000", 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6290000000000001e-06 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636886+0000", 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2199999999999997e-06 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58345 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.642149+0000", 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.001299379, 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011213709000000001, 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58345 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642149+0000", 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642149+0000", 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642148+0000", 2024-09-13T09:18:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642158+0000", 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7429999999999997e-06 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642176+0000", 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7827999999999999e-05 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642183+0000", 2024-09-13T09:18:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9709999999999998e-06 2024-09-13T09:18:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57644 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632466+0000", 2024-09-13T09:18:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010982199999999999, 2024-09-13T09:18:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020994604, 2024-09-13T09:18:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57644 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632466+0000", 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632466+0000", 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632465+0000", 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632468+0000", 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2289999999999999e-06 2024-09-13T09:18:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632476+0000", 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6539999999999999e-06 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632483+0000", 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5080000000000002e-06 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57846 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633441+0000", 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010007546000000001, 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020127250999999999, 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57846 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633441+0000", 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633441+0000", 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633440+0000", 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633446+0000", 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633461+0000", 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5108e-05 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633473+0000", 2024-09-13T09:18:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1994000000000001e-05 2024-09-13T09:18:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57878 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634441+0000", 2024-09-13T09:18:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0090066980000000005, 2024-09-13T09:18:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019211633999999998, 2024-09-13T09:18:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57878 2024-09-13T09:18:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634441+0000", 2024-09-13T09:18:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634441+0000", 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634441+0000", 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634444+0000", 2024-09-13T09:18:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4230000000000001e-06 2024-09-13T09:18:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634453+0000", 2024-09-13T09:18:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5199999999999997e-06 2024-09-13T09:18:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634459+0000", 2024-09-13T09:18:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-13T09:18:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58080 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635316+0000", 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0081322029999999993, 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018413736, 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58080 2024-09-13T09:18:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635316+0000", 2024-09-13T09:18:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635316+0000", 2024-09-13T09:18:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635315+0000", 2024-09-13T09:18:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635318+0000", 2024-09-13T09:18:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.818e-06 2024-09-13T09:18:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635326+0000", 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6369999999999999e-06 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635332+0000", 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2999999999999998e-06 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58112 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636147+0000", 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0073014400000000002, 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017663720000000001, 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58112 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636147+0000", 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636147+0000", 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636146+0000", 2024-09-13T09:18:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636149+0000", 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7939999999999998e-06 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636157+0000", 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636163+0000", 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3289999999999999e-06 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58144 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636891+0000", 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0065568010000000001, 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017044783000000001, 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.046 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58144 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636891+0000", 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636891+0000", 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636891+0000", 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636893+0000", 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6479999999999999e-06 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636901+0000", 2024-09-13T09:18:36.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4970000000000004e-06 2024-09-13T09:18:36.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636907+0000", 2024-09-13T09:18:36.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1029999999999998e-06 2024-09-13T09:18:36.049 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.049 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.049 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.050 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58346 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.050 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.642219+0000", 2024-09-13T09:18:36.050 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.001229071, 2024-09-13T09:18:36.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011795844, 2024-09-13T09:18:36.050 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.050 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.050 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.050 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.050 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.050 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58346 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642219+0000", 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642219+0000", 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642218+0000", 2024-09-13T09:18:36.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642222+0000", 2024-09-13T09:18:36.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2830000000000002e-06 2024-09-13T09:18:36.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642230+0000", 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6910000000000003e-06 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642237+0000", 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6120000000000004e-06 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57645 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.053 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632502+0000", 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010945676999999999, 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021612711999999999, 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57645 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632502+0000", 2024-09-13T09:18:36.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632502+0000", 2024-09-13T09:18:36.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632502+0000", 2024-09-13T09:18:36.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632505+0000", 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1480000000000002e-06 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632513+0000", 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4370000000000007e-06 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632520+0000", 2024-09-13T09:18:36.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.64e-06 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57847 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633483+0000", 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0099650239999999994, 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020746440000000001, 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57847 2024-09-13T09:18:36.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.058 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633483+0000", 2024-09-13T09:18:36.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633483+0000", 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633482+0000", 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633508+0000", 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5559e-05 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633519+0000", 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1455e-05 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633526+0000", 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3250000000000002e-06 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57879 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634495+0000", 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0089534839999999994, 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019819728000000002, 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57879 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634495+0000", 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634495+0000", 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634494+0000", 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634497+0000", 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2830000000000002e-06 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634506+0000", 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4710000000000007e-06 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634512+0000", 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3090000000000001e-06 2024-09-13T09:18:36.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58081 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635337+0000", 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0081110379999999992, 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019054813, 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58081 2024-09-13T09:18:36.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635337+0000", 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635337+0000", 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635337+0000", 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635339+0000", 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8940000000000002e-06 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635347+0000", 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.554e-06 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635353+0000", 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1659999999999999e-06 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58113 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636168+0000", 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0072799800000000001, 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018306491000000001, 2024-09-13T09:18:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58113 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636168+0000", 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636168+0000", 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636168+0000", 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636170+0000", 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-13T09:18:36.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636178+0000", 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4919999999999996e-06 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636184+0000", 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2879999999999998e-06 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58145 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636912+0000", 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0065359670000000002, 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017639798000000002, 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58145 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636912+0000", 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636912+0000", 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636912+0000", 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636914+0000", 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636922+0000", 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6170000000000001e-06 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636928+0000", 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1829999999999999e-06 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58347 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.642243+0000", 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0012054170000000001, 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012390264, 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58347 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642243+0000", 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642243+0000", 2024-09-13T09:18:36.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642242+0000", 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642245+0000", 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8940000000000002e-06 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642253+0000", 2024-09-13T09:18:36.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6109999999999996e-06 2024-09-13T09:18:36.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642259+0000", 2024-09-13T09:18:36.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2949999999999999e-06 2024-09-13T09:18:36.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.077 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57816 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632526+0000", 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010922177, 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022207415000000001, 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57816 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632526+0000", 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632526+0000", 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632525+0000", 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632528+0000", 2024-09-13T09:18:36.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3000000000000002e-06 2024-09-13T09:18:36.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632536+0000", 2024-09-13T09:18:36.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7260000000000002e-06 2024-09-13T09:18:36.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632543+0000", 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3829999999999997e-06 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57848 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633534+0000", 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0099139269999999995, 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021316414999999998, 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57848 2024-09-13T09:18:36.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.082 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633534+0000", 2024-09-13T09:18:36.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633534+0000", 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633533+0000", 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633537+0000", 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7170000000000002e-06 2024-09-13T09:18:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633545+0000", 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9109999999999993e-06 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633552+0000", 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6460000000000004e-06 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57880 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634517+0000", 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0089306360000000005, 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020412032, 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57880 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.085 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634517+0000", 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634517+0000", 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634517+0000", 2024-09-13T09:18:36.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634520+0000", 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634528+0000", 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634546+0000", 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7399999999999999e-05 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58082 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635358+0000", 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0080897609999999991, 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019651653000000002, 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58082 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635358+0000", 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635358+0000", 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635358+0000", 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635361+0000", 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9739999999999998e-06 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635368+0000", 2024-09-13T09:18:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-13T09:18:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635375+0000", 2024-09-13T09:18:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4080000000000003e-06 2024-09-13T09:18:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58114 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636189+0000", 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.007258937, 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018897925999999999, 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58114 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636189+0000", 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636189+0000", 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636189+0000", 2024-09-13T09:18:36.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636192+0000", 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636199+0000", 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5970000000000003e-06 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636206+0000", 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3029999999999996e-06 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58146 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636933+0000", 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0065148640000000004, 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018232132000000002, 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.096 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.096 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58146 2024-09-13T09:18:36.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.096 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636933+0000", 2024-09-13T09:18:36.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636933+0000", 2024-09-13T09:18:36.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636933+0000", 2024-09-13T09:18:36.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636938+0000", 2024-09-13T09:18:36.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636945+0000", 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6979999999999998e-06 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636952+0000", 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5080000000000002e-06 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.099 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58348 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.099 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.642264+0000", 2024-09-13T09:18:36.099 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.001183857, 2024-09-13T09:18:36.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012981535000000001, 2024-09-13T09:18:36.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58348 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642264+0000", 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642264+0000", 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642264+0000", 2024-09-13T09:18:36.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642268+0000", 2024-09-13T09:18:36.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1150000000000004e-06 2024-09-13T09:18:36.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642276+0000", 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7389999999999999e-06 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642283+0000", 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7719999999999994e-06 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57817 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.102 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632548+0000", 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.01090032, 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0227914, 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57817 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632548+0000", 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632548+0000", 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632547+0000", 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632550+0000", 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9569999999999998e-06 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632558+0000", 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6969999999999993e-06 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632564+0000", 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4520000000000002e-06 2024-09-13T09:18:36.105 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.106 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.106 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57849 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.106 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633557+0000", 2024-09-13T09:18:36.106 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0098906610000000002, 2024-09-13T09:18:36.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021897649000000002, 2024-09-13T09:18:36.106 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.106 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.106 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.106 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57849 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633557+0000", 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633557+0000", 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633557+0000", 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633560+0000", 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.332e-06 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633568+0000", 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6709999999999999e-06 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633574+0000", 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57881 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.109 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634582+0000", 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0088656619999999998, 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020951391999999999, 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57881 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634582+0000", 2024-09-13T09:18:36.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634582+0000", 2024-09-13T09:18:36.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634582+0000", 2024-09-13T09:18:36.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634585+0000", 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6169999999999998e-06 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634595+0000", 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5710000000000004e-06 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634602+0000", 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0319999999999998e-06 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58083 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635380+0000", 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0080682210000000004, 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020235973000000001, 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58083 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635380+0000", 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635380+0000", 2024-09-13T09:18:36.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635379+0000", 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635382+0000", 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7709999999999998e-06 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635390+0000", 2024-09-13T09:18:36.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6459999999999993e-06 2024-09-13T09:18:36.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635396+0000", 2024-09-13T09:18:36.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0430000000000004e-06 2024-09-13T09:18:36.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.116 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58115 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636211+0000", 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0072371170000000004, 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019482139999999998, 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58115 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636211+0000", 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636211+0000", 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636210+0000", 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636218+0000", 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0080000000000002e-06 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636229+0000", 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0519999999999999e-05 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636235+0000", 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4200000000000004e-06 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58147 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636957+0000", 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0064911810000000004, 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018816574999999999, 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58147 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636957+0000", 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636957+0000", 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636956+0000", 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636959+0000", 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.785e-06 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636967+0000", 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5660000000000001e-06 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636973+0000", 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58349 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.123 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.642289+0000", 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0011594769999999999, 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013561747000000001, 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58349 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642289+0000", 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642289+0000", 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642288+0000", 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642291+0000", 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9340000000000002e-06 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642299+0000", 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5859999999999999e-06 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642305+0000", 2024-09-13T09:18:36.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5479999999999998e-06 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57818 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632569+0000", 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010878680999999999, 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023379759, 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57818 2024-09-13T09:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632569+0000", 2024-09-13T09:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632569+0000", 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632569+0000", 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632572+0000", 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8909999999999999e-06 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632579+0000", 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6750000000000002e-06 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632586+0000", 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.130 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57850 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633580+0000", 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0098684099999999993, 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022500055000000001, 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57850 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633580+0000", 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633580+0000", 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633579+0000", 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633582+0000", 2024-09-13T09:18:36.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8049999999999998e-06 2024-09-13T09:18:36.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633590+0000", 2024-09-13T09:18:36.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:18:36.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633596+0000", 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3969999999999999e-06 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57882 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634608+0000", 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0088401969999999993, 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021559773000000001, 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.134 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57882 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634608+0000", 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634608+0000", 2024-09-13T09:18:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634607+0000", 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634610+0000", 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0170000000000001e-06 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634618+0000", 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8150000000000006e-06 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634624+0000", 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58084 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.137 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635401+0000", 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0080471759999999996, 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020844231000000001, 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58084 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635401+0000", 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635401+0000", 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635400+0000", 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635403+0000", 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8710000000000001e-06 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635411+0000", 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6539999999999999e-06 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635417+0000", 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2519999999999996e-06 2024-09-13T09:18:36.140 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.141 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.141 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.141 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58116 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.141 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636240+0000", 2024-09-13T09:18:36.141 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0072077690000000002, 2024-09-13T09:18:36.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0200813, 2024-09-13T09:18:36.141 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.141 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58116 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636240+0000", 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636240+0000", 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636240+0000", 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636243+0000", 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9569999999999998e-06 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636250+0000", 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636257+0000", 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.302e-06 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58148 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636980+0000", 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0064680420000000002, 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019422347, 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58148 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636980+0000", 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636980+0000", 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636980+0000", 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636982+0000", 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.774e-06 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636990+0000", 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6660000000000008e-06 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636996+0000", 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3459999999999999e-06 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58350 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.642310+0000", 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0011377290000000001, 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014168195999999999, 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58350 2024-09-13T09:18:36.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.149 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642310+0000", 2024-09-13T09:18:36.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642310+0000", 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642310+0000", 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642313+0000", 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9450000000000002e-06 2024-09-13T09:18:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642320+0000", 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5569999999999998e-06 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642327+0000", 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2230000000000004e-06 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57819 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632614+0000", 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010834124000000001, 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023962139, 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57819 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632614+0000", 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632614+0000", 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632613+0000", 2024-09-13T09:18:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632617+0000", 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6480000000000001e-06 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632626+0000", 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6230000000000003e-06 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632632+0000", 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.494e-06 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57851 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633605+0000", 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0098432480000000006, 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023086995999999999, 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.155 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57851 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633605+0000", 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633605+0000", 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633604+0000", 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633607+0000", 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3830000000000001e-06 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633615+0000", 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7999999999999999e-06 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633622+0000", 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.443e-06 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.158 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57883 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634630+0000", 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0088183889999999994, 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022189561999999999, 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57883 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634630+0000", 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634630+0000", 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634629+0000", 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634632+0000", 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1920000000000001e-06 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634640+0000", 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7740000000000004e-06 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634647+0000", 2024-09-13T09:18:36.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3910000000000003e-06 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58085 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635454+0000", 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0079944679999999994, 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021443530999999998, 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58085 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635454+0000", 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635454+0000", 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635453+0000", 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635456+0000", 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.551e-06 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635473+0000", 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6849000000000001e-05 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635480+0000", 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58117 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.165 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636262+0000", 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0071862840000000002, 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020712017999999999, 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58117 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636262+0000", 2024-09-13T09:18:36.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636262+0000", 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636261+0000", 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636264+0000", 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636272+0000", 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6629999999999993e-06 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636278+0000", 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2230000000000004e-06 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.168 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58149 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637003+0000", 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0064447849999999998, 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.02004769, 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58149 2024-09-13T09:18:36.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637003+0000", 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637003+0000", 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637003+0000", 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637006+0000", 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637013+0000", 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7430000000000002e-06 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637020+0000", 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3539999999999996e-06 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58351 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.642332+0000", 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0011161490000000001, 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014795981999999999, 2024-09-13T09:18:36.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58351 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642332+0000", 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642332+0000", 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642331+0000", 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642335+0000", 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2339999999999999e-06 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642342+0000", 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5689999999999999e-06 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642348+0000", 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57820 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632671+0000", 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010776956000000001, 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024551990999999999, 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57820 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632671+0000", 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632671+0000", 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632670+0000", 2024-09-13T09:18:36.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632674+0000", 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1990000000000002e-06 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632682+0000", 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5799999999999992e-06 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632689+0000", 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3860000000000004e-06 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57852 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633627+0000", 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0098209909999999994, 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023705364, 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.179 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57852 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633627+0000", 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633627+0000", 2024-09-13T09:18:36.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633627+0000", 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633629+0000", 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8710000000000001e-06 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633637+0000", 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6290000000000001e-06 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633644+0000", 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4710000000000004e-06 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57884 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634652+0000", 2024-09-13T09:18:36.182 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0087958169999999992, 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022760235, 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57884 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634652+0000", 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634652+0000", 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634652+0000", 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634660+0000", 2024-09-13T09:18:36.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5169999999999999e-06 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634668+0000", 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634675+0000", 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6540000000000002e-06 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58086 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635486+0000", 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0079625649999999996, 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022003811000000002, 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58086 2024-09-13T09:18:36.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635486+0000", 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635486+0000", 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635485+0000", 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635488+0000", 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2679999999999999e-06 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635496+0000", 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.926e-06 2024-09-13T09:18:36.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635502+0000", 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1709999999999999e-06 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58118 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636284+0000", 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0071644919999999997, 2024-09-13T09:18:36.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021284389000000001, 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58118 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636284+0000", 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636284+0000", 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636283+0000", 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636286+0000", 2024-09-13T09:18:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9229999999999998e-06 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636294+0000", 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6769999999999995e-06 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636300+0000", 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.229e-06 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.192 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58150 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637025+0000", 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.006423242, 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020620887000000001, 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58150 2024-09-13T09:18:36.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637025+0000", 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637025+0000", 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637024+0000", 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637027+0000", 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.74e-06 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637035+0000", 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6079999999999998e-06 2024-09-13T09:18:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637041+0000", 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2140000000000001e-06 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58352 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.645994+0000", 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9974537, 2024-09-13T09:18:36.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011731139999999999, 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58352 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.645994+0000", 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.645994+0000", 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.642353+0000", 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9963584 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646005+0000", 2024-09-13T09:18:36.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003652218 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646032+0000", 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6737000000000002e-05 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646049+0000", 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7626e-05 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57821 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.632694+0000", 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.010754019, 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025125074000000001, 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57821 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.200 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632694+0000", 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632694+0000", 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632694+0000", 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632696+0000", 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8909999999999999e-06 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632704+0000", 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8599999999999993e-06 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.632711+0000", 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.409e-06 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57853 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633650+0000", 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0097977020000000001, 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024302720999999999, 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57853 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633650+0000", 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633650+0000", 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633650+0000", 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633653+0000", 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.937e-06 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633660+0000", 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633667+0000", 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2539999999999998e-06 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57885 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634680+0000", 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.008767703, 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023355793, 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57885 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634680+0000", 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634680+0000", 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634680+0000", 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634683+0000", 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9220000000000001e-06 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634690+0000", 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6660000000000008e-06 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634697+0000", 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4570000000000002e-06 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58087 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635508+0000", 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0079405880000000002, 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022604814000000001, 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58087 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635508+0000", 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635508+0000", 2024-09-13T09:18:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635507+0000", 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635510+0000", 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635517+0000", 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6140000000000003e-06 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635524+0000", 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1e-06 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58119 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636305+0000", 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0071432839999999997, 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021886211999999999, 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58119 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636305+0000", 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636305+0000", 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636304+0000", 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636307+0000", 2024-09-13T09:18:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:18:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636315+0000", 2024-09-13T09:18:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6709999999999999e-06 2024-09-13T09:18:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636321+0000", 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3749999999999999e-06 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58151 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637046+0000", 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0064020650000000002, 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021221660999999999, 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58151 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637046+0000", 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637046+0000", 2024-09-13T09:18:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637045+0000", 2024-09-13T09:18:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637052+0000", 2024-09-13T09:18:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8580000000000002e-06 2024-09-13T09:18:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637060+0000", 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9419999999999995e-06 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637067+0000", 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.392e-06 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58353 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.646065+0000", 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9973826, 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012283433, 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58353 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646065+0000", 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646065+0000", 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646064+0000", 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646073+0000", 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6910000000000003e-06 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646091+0000", 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646110+0000", 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8527999999999999e-05 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57854 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633673+0000", 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0097747570000000002, 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024770192, 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57854 2024-09-13T09:18:36.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.225 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633673+0000", 2024-09-13T09:18:36.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633673+0000", 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633673+0000", 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633676+0000", 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:18:36.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633683+0000", 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8050000000000007e-06 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633690+0000", 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6089999999999997e-06 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57886 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634703+0000", 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0087453860000000008, 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023847253999999998, 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57886 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.228 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634703+0000", 2024-09-13T09:18:36.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634703+0000", 2024-09-13T09:18:36.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634702+0000", 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634705+0000", 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.157e-06 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634713+0000", 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5340000000000002e-06 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634719+0000", 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5169999999999996e-06 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58088 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635529+0000", 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0079192050000000003, 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023098858, 2024-09-13T09:18:36.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.232 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58088 2024-09-13T09:18:36.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.232 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635529+0000", 2024-09-13T09:18:36.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635529+0000", 2024-09-13T09:18:36.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635528+0000", 2024-09-13T09:18:36.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635531+0000", 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8449999999999999e-06 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635539+0000", 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6090000000000003e-06 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635545+0000", 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1249999999999998e-06 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58120 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636326+0000", 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0071221360000000003, 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022378441999999998, 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58120 2024-09-13T09:18:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636326+0000", 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636326+0000", 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636325+0000", 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636328+0000", 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9280000000000002e-06 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636336+0000", 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5660000000000001e-06 2024-09-13T09:18:36.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636342+0000", 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2140000000000001e-06 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58152 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637072+0000", 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0063761080000000001, 2024-09-13T09:18:36.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021713488, 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58152 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637072+0000", 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637072+0000", 2024-09-13T09:18:36.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637071+0000", 2024-09-13T09:18:36.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637074+0000", 2024-09-13T09:18:36.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8119999999999999e-06 2024-09-13T09:18:36.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637082+0000", 2024-09-13T09:18:36.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5510000000000002e-06 2024-09-13T09:18:36.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637088+0000", 2024-09-13T09:18:36.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:18:36.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.242 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.242 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58354 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.242 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.646121+0000", 2024-09-13T09:18:36.242 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9973269, 2024-09-13T09:18:36.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012742454, 2024-09-13T09:18:36.242 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.242 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58354 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646121+0000", 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646121+0000", 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646121+0000", 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646126+0000", 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4280000000000004e-06 2024-09-13T09:18:36.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646143+0000", 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7048999999999999e-05 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646156+0000", 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3037000000000001e-05 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57855 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633696+0000", 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0097523939999999993, 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025263266999999999, 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57855 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633696+0000", 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633696+0000", 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633695+0000", 2024-09-13T09:18:36.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633698+0000", 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0769999999999999e-06 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633706+0000", 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6650000000000003e-06 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633712+0000", 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2949999999999999e-06 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57887 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634738+0000", 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0087101520000000005, 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024323551999999998, 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57887 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634738+0000", 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634738+0000", 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634737+0000", 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634741+0000", 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634749+0000", 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6770000000000001e-06 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634756+0000", 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4339999999999997e-06 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58089 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635550+0000", 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0078979740000000003, 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023588967999999998, 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58089 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635550+0000", 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635550+0000", 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635550+0000", 2024-09-13T09:18:36.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635552+0000", 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.791e-06 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635560+0000", 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4939999999999997e-06 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635566+0000", 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.195e-06 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58121 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636347+0000", 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0071009159999999997, 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022868825999999998, 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58121 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636347+0000", 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636347+0000", 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636347+0000", 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636349+0000", 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.774e-06 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636357+0000", 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6059999999999997e-06 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636363+0000", 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2419999999999997e-06 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58153 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637094+0000", 2024-09-13T09:18:36.260 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0063544680000000003, 2024-09-13T09:18:36.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0221996, 2024-09-13T09:18:36.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.260 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58153 2024-09-13T09:18:36.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.260 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637094+0000", 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637094+0000", 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637093+0000", 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637096+0000", 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.971e-06 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637103+0000", 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6279999999999996e-06 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637110+0000", 2024-09-13T09:18:36.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2920000000000001e-06 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58355 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.646167+0000", 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9972806, 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01320353, 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.264 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58355 2024-09-13T09:18:36.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.264 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646167+0000", 2024-09-13T09:18:36.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646167+0000", 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646166+0000", 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646172+0000", 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.66e-06 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646189+0000", 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7416999999999999e-05 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646203+0000", 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3226e-05 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57856 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633747+0000", 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0097008200000000006, 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025718135999999999, 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57856 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633747+0000", 2024-09-13T09:18:36.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633747+0000", 2024-09-13T09:18:36.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633747+0000", 2024-09-13T09:18:36.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633750+0000", 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5549999999999998e-06 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633759+0000", 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9400000000000008e-06 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633766+0000", 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5339999999999996e-06 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57888 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634794+0000", 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0086541559999999997, 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024775040000000002, 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57888 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634794+0000", 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634794+0000", 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634793+0000", 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634797+0000", 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2619999999999999e-06 2024-09-13T09:18:36.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634805+0000", 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5750000000000001e-06 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634812+0000", 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3879999999999997e-06 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58090 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635571+0000", 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0078768520000000002, 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024076200999999998, 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58090 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635571+0000", 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635571+0000", 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635571+0000", 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635579+0000", 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8259999999999993e-06 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635586+0000", 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9459999999999998e-06 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635593+0000", 2024-09-13T09:18:36.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4049999999999997e-06 2024-09-13T09:18:36.277 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.277 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.277 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.277 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58122 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.277 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636368+0000", 2024-09-13T09:18:36.277 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0070796820000000003, 2024-09-13T09:18:36.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.02335605, 2024-09-13T09:18:36.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58122 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636368+0000", 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636368+0000", 2024-09-13T09:18:36.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636368+0000", 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636371+0000", 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.729e-06 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636378+0000", 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.542e-06 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636384+0000", 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3119999999999999e-06 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.280 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58154 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637136+0000", 2024-09-13T09:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0063124399999999999, 2024-09-13T09:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022667876, 2024-09-13T09:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58154 2024-09-13T09:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637136+0000", 2024-09-13T09:18:36.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637136+0000", 2024-09-13T09:18:36.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637135+0000", 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637138+0000", 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.089e-06 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637147+0000", 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6829999999999997e-06 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637153+0000", 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3369999999999996e-06 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58356 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.646213+0000", 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9972348, 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013667903, 2024-09-13T09:18:36.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58356 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646213+0000", 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646213+0000", 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646213+0000", 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646218+0000", 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1850000000000003e-06 2024-09-13T09:18:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646235+0000", 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6765999999999998e-05 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646248+0000", 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2957e-05 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57857 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633771+0000", 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0096768659999999992, 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026203568999999999, 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57857 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633771+0000", 2024-09-13T09:18:36.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633771+0000", 2024-09-13T09:18:36.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633771+0000", 2024-09-13T09:18:36.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633774+0000", 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1489999999999998e-06 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633782+0000", 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7680000000000008e-06 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633788+0000", 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.263e-06 2024-09-13T09:18:36.290 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57889 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634817+0000", 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0086310099999999997, 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025259508999999999, 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57889 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634817+0000", 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634817+0000", 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634817+0000", 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634819+0000", 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634827+0000", 2024-09-13T09:18:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7279999999999995e-06 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634834+0000", 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4629999999999998e-06 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58091 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635598+0000", 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0078498230000000006, 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024556461000000002, 2024-09-13T09:18:36.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58091 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635598+0000", 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635598+0000", 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635598+0000", 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635600+0000", 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7599999999999998e-06 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635608+0000", 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5939999999999996e-06 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635614+0000", 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2140000000000001e-06 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58123 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.297 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636391+0000", 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0070569990000000004, 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023840871, 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58123 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636391+0000", 2024-09-13T09:18:36.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636391+0000", 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636391+0000", 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636393+0000", 2024-09-13T09:18:36.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.88e-06 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636401+0000", 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5970000000000003e-06 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636408+0000", 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3649999999999998e-06 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.301 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58155 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.301 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637159+0000", 2024-09-13T09:18:36.301 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0062893580000000001, 2024-09-13T09:18:36.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023150381000000001, 2024-09-13T09:18:36.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.304 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.304 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.304 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.304 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.304 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58155 2024-09-13T09:18:36.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637159+0000", 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637159+0000", 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637158+0000", 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637161+0000", 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.903e-06 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637169+0000", 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637175+0000", 2024-09-13T09:18:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:18:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58357 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.646258+0000", 2024-09-13T09:18:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9971895, 2024-09-13T09:18:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014128786000000001, 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58357 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646258+0000", 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646258+0000", 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646258+0000", 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646263+0000", 2024-09-13T09:18:36.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646280+0000", 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.668e-05 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646292+0000", 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2602000000000001e-05 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57858 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633793+0000", 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0096548269999999995, 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026688488, 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57858 2024-09-13T09:18:36.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.312 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633793+0000", 2024-09-13T09:18:36.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633793+0000", 2024-09-13T09:18:36.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633793+0000", 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633796+0000", 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8890000000000002e-06 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633803+0000", 2024-09-13T09:18:36.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6629999999999993e-06 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633810+0000", 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3199999999999996e-06 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57890 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634840+0000", 2024-09-13T09:18:36.314 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0086076650000000005, 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025745958999999999, 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57890 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634840+0000", 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634840+0000", 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634840+0000", 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634843+0000", 2024-09-13T09:18:36.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.112e-06 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634850+0000", 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634857+0000", 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3509999999999998e-06 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58092 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635621+0000", 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0078272089999999999, 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025043788000000001, 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58092 2024-09-13T09:18:36.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635621+0000", 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635621+0000", 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635620+0000", 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635623+0000", 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.937e-06 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635631+0000", 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7800000000000001e-06 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635637+0000", 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3799999999999999e-06 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58124 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636414+0000", 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0070344479999999996, 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024328026999999999, 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58124 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636414+0000", 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636414+0000", 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636413+0000", 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636416+0000", 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.914e-06 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636424+0000", 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6140000000000003e-06 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636430+0000", 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2489999999999998e-06 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58156 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637180+0000", 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0062679839999999999, 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023640405, 2024-09-13T09:18:36.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58156 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637180+0000", 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637180+0000", 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637180+0000", 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637182+0000", 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.897e-06 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637192+0000", 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0459999999999994e-06 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637199+0000", 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7570000000000004e-06 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58358 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.646317+0000", 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9971313, 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014581751, 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58358 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646317+0000", 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646317+0000", 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646316+0000", 2024-09-13T09:18:36.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646322+0000", 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8969999999999996e-06 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646361+0000", 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9031e-05 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646376+0000", 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5299999999999999e-05 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57859 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633815+0000", 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0096328609999999995, 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027176922999999999, 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57859 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633815+0000", 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633815+0000", 2024-09-13T09:18:36.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633814+0000", 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633818+0000", 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0800000000000002e-06 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633839+0000", 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1101999999999999e-05 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633849+0000", 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0572e-05 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57891 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634862+0000", 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0085856739999999997, 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026234838, 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57891 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634862+0000", 2024-09-13T09:18:36.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634862+0000", 2024-09-13T09:18:36.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634862+0000", 2024-09-13T09:18:36.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634865+0000", 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8370000000000001e-06 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634872+0000", 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5660000000000001e-06 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634878+0000", 2024-09-13T09:18:36.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2999999999999998e-06 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58093 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635643+0000", 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.007805106, 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025532043000000001, 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58093 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635643+0000", 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635643+0000", 2024-09-13T09:18:36.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635642+0000", 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635645+0000", 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0230000000000001e-06 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635653+0000", 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.588e-06 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635661+0000", 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5969999999999992e-06 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58125 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.341 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636435+0000", 2024-09-13T09:18:36.342 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0070130990000000001, 2024-09-13T09:18:36.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024816920999999999, 2024-09-13T09:18:36.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.342 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58125 2024-09-13T09:18:36.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.342 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636435+0000", 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636435+0000", 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636434+0000", 2024-09-13T09:18:36.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636437+0000", 2024-09-13T09:18:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.96e-06 2024-09-13T09:18:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636445+0000", 2024-09-13T09:18:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5830000000000001e-06 2024-09-13T09:18:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636451+0000", 2024-09-13T09:18:36.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3570000000000003e-06 2024-09-13T09:18:36.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.345 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.345 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58157 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.345 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637205+0000", 2024-09-13T09:18:36.345 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0062436089999999998, 2024-09-13T09:18:36.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024126504999999999, 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58157 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637205+0000", 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637205+0000", 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637204+0000", 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637207+0000", 2024-09-13T09:18:36.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8229999999999999e-06 2024-09-13T09:18:36.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637214+0000", 2024-09-13T09:18:36.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5859999999999999e-06 2024-09-13T09:18:36.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637221+0000", 2024-09-13T09:18:36.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3940000000000001e-06 2024-09-13T09:18:36.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.348 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58359 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.646388+0000", 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9970598, 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015021132, 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58359 2024-09-13T09:18:36.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646388+0000", 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646388+0000", 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646387+0000", 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646393+0000", 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8860000000000001e-06 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646412+0000", 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9042000000000002e-05 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646431+0000", 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8926000000000001e-05 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57860 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633873+0000", 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0095751359999999997, 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.02763057, 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57860 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633873+0000", 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633873+0000", 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633872+0000", 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633877+0000", 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633887+0000", 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8199999999999992e-06 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633894+0000", 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1350000000000003e-06 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57892 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634884+0000", 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0085644510000000007, 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026725973, 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57892 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634884+0000", 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634884+0000", 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634883+0000", 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634886+0000", 2024-09-13T09:18:36.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.88e-06 2024-09-13T09:18:36.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634895+0000", 2024-09-13T09:18:36.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2050000000000008e-06 2024-09-13T09:18:36.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634903+0000", 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0779999999999996e-06 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58094 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635676+0000", 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0077723150000000001, 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026011822, 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58094 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635676+0000", 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635676+0000", 2024-09-13T09:18:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635675+0000", 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635678+0000", 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0520000000000002e-06 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635686+0000", 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1850000000000004e-06 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635693+0000", 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6769999999999998e-06 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58126 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.362 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636457+0000", 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0069915400000000001, 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025307568999999999, 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58126 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636457+0000", 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636457+0000", 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636456+0000", 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636459+0000", 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9059999999999998e-06 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636467+0000", 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6399999999999997e-06 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636473+0000", 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3219999999999998e-06 2024-09-13T09:18:36.365 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58328 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637226+0000", 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0062222470000000002, 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024655149000000001, 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.366 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58328 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637226+0000", 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637226+0000", 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637225+0000", 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637228+0000", 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8399999999999999e-06 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637236+0000", 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4880000000000001e-06 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637242+0000", 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58360 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.646443+0000", 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.997005, 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015521350999999999, 2024-09-13T09:18:36.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58360 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646443+0000", 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646443+0000", 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646442+0000", 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646448+0000", 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:18:36.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646464+0000", 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.626e-05 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646477+0000", 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3103e-05 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57861 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633900+0000", 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0095484760000000002, 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028160236000000002, 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57861 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633900+0000", 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633900+0000", 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633899+0000", 2024-09-13T09:18:36.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633902+0000", 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0460000000000001e-06 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633910+0000", 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8599999999999993e-06 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633917+0000", 2024-09-13T09:18:36.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5540000000000003e-06 2024-09-13T09:18:36.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.376 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.376 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57893 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.376 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634909+0000", 2024-09-13T09:18:36.376 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0085395309999999995, 2024-09-13T09:18:36.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027262947999999999, 2024-09-13T09:18:36.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57893 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634909+0000", 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634909+0000", 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634908+0000", 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634911+0000", 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.774e-06 2024-09-13T09:18:36.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634920+0000", 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8799999999999997e-06 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634926+0000", 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2750000000000001e-06 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58095 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635699+0000", 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0077491929999999997, 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026550995000000001, 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58095 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635699+0000", 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635699+0000", 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635698+0000", 2024-09-13T09:18:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635701+0000", 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.92e-06 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635709+0000", 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6399999999999997e-06 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635715+0000", 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3029999999999996e-06 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58127 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636478+0000", 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0069697910000000004, 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025848143000000001, 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.383 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58127 2024-09-13T09:18:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636478+0000", 2024-09-13T09:18:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636478+0000", 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636478+0000", 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636486+0000", 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8480000000000001e-06 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636494+0000", 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0229999999999992e-06 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636500+0000", 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5660000000000003e-06 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.386 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58329 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.387 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637247+0000", 2024-09-13T09:18:36.387 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0062006639999999998, 2024-09-13T09:18:36.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025155713, 2024-09-13T09:18:36.387 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58329 2024-09-13T09:18:36.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.387 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637247+0000", 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637247+0000", 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637247+0000", 2024-09-13T09:18:36.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637250+0000", 2024-09-13T09:18:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.672e-06 2024-09-13T09:18:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637257+0000", 2024-09-13T09:18:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5680000000000002e-06 2024-09-13T09:18:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637264+0000", 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6540000000000002e-06 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58361 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.646489+0000", 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9969597, 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015992577000000001, 2024-09-13T09:18:36.390 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58361 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646489+0000", 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646489+0000", 2024-09-13T09:18:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646487+0000", 2024-09-13T09:18:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646493+0000", 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7910000000000001e-06 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646527+0000", 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3488000000000002e-05 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646544+0000", 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7025999999999999e-05 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57862 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633947+0000", 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0095011939999999993, 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028629293, 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57862 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633947+0000", 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633947+0000", 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633946+0000", 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633950+0000", 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4000000000000002e-06 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633959+0000", 2024-09-13T09:18:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633966+0000", 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9820000000000002e-06 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57894 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634933+0000", 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0085153629999999998, 2024-09-13T09:18:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027767377999999999, 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57894 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634933+0000", 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634933+0000", 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634932+0000", 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634935+0000", 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7659999999999999e-06 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634944+0000", 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1060000000000006e-06 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634950+0000", 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3450000000000002e-06 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58096 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635720+0000", 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0077276869999999996, 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027058685999999998, 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58096 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635720+0000", 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635720+0000", 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635720+0000", 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635723+0000", 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.937e-06 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635730+0000", 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5020000000000003e-06 2024-09-13T09:18:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635737+0000", 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3749999999999999e-06 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58128 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636506+0000", 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0069424860000000003, 2024-09-13T09:18:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026350098999999998, 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58128 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636506+0000", 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636506+0000", 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636505+0000", 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636508+0000", 2024-09-13T09:18:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.875e-06 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636517+0000", 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2939999999999994e-06 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636523+0000", 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58330 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637269+0000", 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0061791440000000001, 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025663525, 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58330 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637269+0000", 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637269+0000", 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637268+0000", 2024-09-13T09:18:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637271+0000", 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7480000000000002e-06 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637280+0000", 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.02e-06 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637287+0000", 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4710000000000004e-06 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58362 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.646555+0000", 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9968929, 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016458782000000002, 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58362 2024-09-13T09:18:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646555+0000", 2024-09-13T09:18:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646555+0000", 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646554+0000", 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646560+0000", 2024-09-13T09:18:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1720000000000004e-06 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646577+0000", 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7082e-05 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646591+0000", 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3295000000000001e-05 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57863 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.633972+0000", 2024-09-13T09:18:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0094762200000000005, 2024-09-13T09:18:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029136548000000002, 2024-09-13T09:18:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57863 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633972+0000", 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633972+0000", 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633971+0000", 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633974+0000", 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633982+0000", 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7370000000000006e-06 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.633988+0000", 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:57895 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.634956+0000", 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0084924860000000005, 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028275723999999999, 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 57895 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634956+0000", 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634956+0000", 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634955+0000", 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634958+0000", 2024-09-13T09:18:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8890000000000002e-06 2024-09-13T09:18:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634966+0000", 2024-09-13T09:18:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7049999999999999e-06 2024-09-13T09:18:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.634972+0000", 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2369999999999997e-06 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58097 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.635742+0000", 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0077061389999999999, 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027567325, 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58097 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635742+0000", 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635742+0000", 2024-09-13T09:18:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635741+0000", 2024-09-13T09:18:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635744+0000", 2024-09-13T09:18:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8310000000000001e-06 2024-09-13T09:18:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635752+0000", 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5109999999999997e-06 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.635758+0000", 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3799999999999999e-06 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58129 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.636529+0000", 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0069193259999999999, 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026857397000000002, 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58129 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636529+0000", 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636529+0000", 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636528+0000", 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636531+0000", 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636539+0000", 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8599999999999993e-06 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.636545+0000", 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3720000000000001e-06 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58331 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.637292+0000", 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0061563900000000003, 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026174569000000002, 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58331 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637292+0000", 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637292+0000", 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637291+0000", 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637294+0000", 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8059999999999999e-06 2024-09-13T09:18:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637311+0000", 2024-09-13T09:18:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6711e-05 2024-09-13T09:18:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.637317+0000", 2024-09-13T09:18:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7349999999999999e-06 2024-09-13T09:18:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:58363 3.a 3.2a30c64a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.646601+0000", 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9968467, 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016943283999999999, 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 58363 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646601+0000", 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646601+0000", 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646600+0000", 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646606+0000", 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646627+0000", 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1506e-05 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.646640+0000", 2024-09-13T09:18:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2836999999999999e-05 2024-09-13T09:18:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-13T09:18:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 218 2024-09-13T09:18:36.436 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:36.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:18:36.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:36.440 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:36.440 INFO:teuthology.orchestra.run.smithi032.stdout:3796 write 0xa709f4d thru 0xa718f1a (0xefce bytes) 2024-09-13T09:18:36.440 INFO:teuthology.orchestra.run.smithi032.stdout:3798 write 0x591bc47 thru 0x591e2ec (0x26a6 bytes) 2024-09-13T09:18:36.440 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:36.440 INFO:teuthology.orchestra.run.smithi032.stdout:3801 trunc from 0xa718f1b to 0x94e8548 2024-09-13T09:18:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:3802 writesame 0xe72ef1 thru 0xe7ef05 data_size 0x25(0xc015 bytes) 2024-09-13T09:18:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:3803 punch from 0x4d902bd to 0x4d9affa, (0xad3d bytes) 2024-09-13T09:18:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:3806 writesame 0x7f84c27 thru 0x7f85313 data_size 0x3(0x6ed bytes) 2024-09-13T09:18:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:3807 trunc from 0x94e8548 to 0xd28fd5a 2024-09-13T09:18:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:3808 trunc from 0xd28fd5a to 0xcfb04a0 2024-09-13T09:18:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:3809 read 0xbec5152 thru 0xbec7b33 (0x29e2 bytes) 2024-09-13T09:18:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:3811 trunc from 0xcfb04a0 to 0x9a6cea 2024-09-13T09:18:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:3813 clone 27 order 18 su 2048 sc 6 2024-09-13T09:18:36.442 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:18:36.443 INFO:teuthology.orchestra.run.smithi032.stdout:flattening image image_client.0-clone26 2024-09-13T09:18:36.443 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-13T09:18:36.443 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:36.434+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:36.539 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:36.534+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:36.578 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:36.579 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:36.579 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:36.579 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:36.579 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:36.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:36.639 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:36.638+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:36.650 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:36.667 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:36.667 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:36.685 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:36.704 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:36.704 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:36.723 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:36.724 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:36.740 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:36.738+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:36.743 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:36.764 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:36.764 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:36.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:36.765 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:36.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.765 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:50082 3.10 3:0ec43ecb:::rbd_object_map.1185172eb410:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T09:18:36.765 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:36.765 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.3985605459999997, 2024-09-13T09:18:36.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80943783999999996, 2024-09-13T09:18:36.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:36.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 50082 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362619+0000", 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362632+0000", 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3338e-05 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362657+0000", 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5207000000000001e-05 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362669+0000", 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1768e-05 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362738+0000", 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9486999999999994e-05 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362748+0000", 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0010000000000001e-05 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169473+0000", 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80672482300000004 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169884+0000", 2024-09-13T09:18:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041114299999999999 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.170206+0000", 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032127700000000002 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171827+0000", 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016217880000000001 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171871+0000", 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171967+0000", 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5752000000000001e-05 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.172059+0000", 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2082000000000004e-05 2024-09-13T09:18:36.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.772 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.772 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.772 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56772 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 4112384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.772 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:36.772 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3308724270000001, 2024-09-13T09:18:36.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054477114, 2024-09-13T09:18:36.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:36.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56772 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430309+0000", 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430312+0000", 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.255e-06 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430319+0000", 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4760000000000001e-06 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430324+0000", 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9799999999999998e-06 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484634+0000", 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054310031000000002 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484697+0000", 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3355999999999994e-05 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484787+0000", 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9350000000000003e-05 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.776 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56785 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5390336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3270889450000001, 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054655007999999998, 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56785 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434094+0000", 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.471e-06 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434097+0000", 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0740000000000001e-06 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434100+0000", 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7829999999999999e-06 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488422+0000", 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054321855000000002 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488563+0000", 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014154 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488748+0000", 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018461399999999999 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56786 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3270795630000001, 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054950539999999999, 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.781 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56786 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434104+0000", 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4020000000000001e-06 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434107+0000", 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.072e-06 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434109+0000", 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.717e-06 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488795+0000", 2024-09-13T09:18:36.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054685661000000003 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488869+0000", 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4265999999999997e-05 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489053+0000", 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018374299999999999 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56787 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5586944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3270693040000001, 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055115725999999997, 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56787 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.786 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434112+0000", 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434114+0000", 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.308e-06 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434117+0000", 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434119+0000", 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.588e-06 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489097+0000", 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054977824000000002 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489146+0000", 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8503e-05 2024-09-13T09:18:36.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489228+0000", 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2694000000000005e-05 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56788 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3270604129999999, 2024-09-13T09:18:36.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055437562000000003, 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56788 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434121+0000", 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434123+0000", 2024-09-13T09:18:36.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2190000000000001e-06 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434126+0000", 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434128+0000", 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8250000000000001e-06 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489293+0000", 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055164092999999997 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489428+0000", 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013572599999999999 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489559+0000", 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013086 2024-09-13T09:18:36.794 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56789 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3270512640000001, 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055674077000000002, 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.795 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56789 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434132+0000", 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.302e-06 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434136+0000", 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434139+0000", 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489583+0000", 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055443994000000003 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489699+0000", 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116611 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489805+0000", 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105331 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.799 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56790 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3270408890000001, 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055716839999999997, 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56790 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434142+0000", 2024-09-13T09:18:36.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2559999999999999e-06 2024-09-13T09:18:36.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434145+0000", 2024-09-13T09:18:36.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.96e-06 2024-09-13T09:18:36.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434148+0000", 2024-09-13T09:18:36.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6479999999999999e-06 2024-09-13T09:18:36.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489851+0000", 2024-09-13T09:18:36.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055702792000000001 2024-09-13T09:18:36.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489858+0000", 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3900000000000004e-06 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56791 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3270321169999999, 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055740029000000003, 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.804 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56791 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434151+0000", 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:18:36.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434154+0000", 2024-09-13T09:18:36.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0139999999999999e-06 2024-09-13T09:18:36.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434157+0000", 2024-09-13T09:18:36.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6699999999999998e-06 2024-09-13T09:18:36.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489884+0000", 2024-09-13T09:18:36.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055727511 2024-09-13T09:18:36.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489890+0000", 2024-09-13T09:18:36.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7200000000000003e-06 2024-09-13T09:18:36.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56792 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3270231880000001, 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055760363, 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56792 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:36.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434163+0000", 2024-09-13T09:18:36.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-13T09:18:36.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434167+0000", 2024-09-13T09:18:36.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.292e-06 2024-09-13T09:18:36.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434169+0000", 2024-09-13T09:18:36.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:18:36.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489914+0000", 2024-09-13T09:18:36.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055744415999999998 2024-09-13T09:18:36.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489919+0000", 2024-09-13T09:18:36.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:18:36.814 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.814 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.814 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.814 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56793 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.814 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:36.815 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3270096680000001, 2024-09-13T09:18:36.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055774049999999999, 2024-09-13T09:18:36.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.815 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.815 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.815 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.815 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56793 2024-09-13T09:18:36.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.816 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:36.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:36.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:36.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434173+0000", 2024-09-13T09:18:36.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-06 2024-09-13T09:18:36.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434177+0000", 2024-09-13T09:18:36.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1070000000000001e-06 2024-09-13T09:18:36.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434179+0000", 2024-09-13T09:18:36.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.666e-06 2024-09-13T09:18:36.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489941+0000", 2024-09-13T09:18:36.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055762071000000003 2024-09-13T09:18:36.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489946+0000", 2024-09-13T09:18:36.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:18:36.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.821 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.821 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.821 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56794 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.821 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:36.821 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3270005600000001, 2024-09-13T09:18:36.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055791704999999997, 2024-09-13T09:18:36.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.822 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56794 2024-09-13T09:18:36.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.822 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:36.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:36.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:36.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434182+0000", 2024-09-13T09:18:36.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1629999999999999e-06 2024-09-13T09:18:36.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434185+0000", 2024-09-13T09:18:36.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9950000000000001e-06 2024-09-13T09:18:36.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434188+0000", 2024-09-13T09:18:36.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:18:36.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489968+0000", 2024-09-13T09:18:36.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055779789000000003 2024-09-13T09:18:36.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489973+0000", 2024-09-13T09:18:36.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2800000000000003e-06 2024-09-13T09:18:36.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.827 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.828 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56795 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.828 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:36.828 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3269916239999999, 2024-09-13T09:18:36.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055809612000000001, 2024-09-13T09:18:36.828 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.828 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.828 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.829 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56795 2024-09-13T09:18:36.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.829 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:36.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:36.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:36.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434191+0000", 2024-09-13T09:18:36.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.212e-06 2024-09-13T09:18:36.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434194+0000", 2024-09-13T09:18:36.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.078e-06 2024-09-13T09:18:36.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434198+0000", 2024-09-13T09:18:36.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9820000000000002e-06 2024-09-13T09:18:36.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489995+0000", 2024-09-13T09:18:36.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055796363000000002 2024-09-13T09:18:36.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490000+0000", 2024-09-13T09:18:36.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1839999999999998e-06 2024-09-13T09:18:36.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.837 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.837 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56796 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.838 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:36.838 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.326981363, 2024-09-13T09:18:36.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055825859999999998, 2024-09-13T09:18:36.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56796 2024-09-13T09:18:36.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.840 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:36.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:36.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434200+0000", 2024-09-13T09:18:36.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434202+0000", 2024-09-13T09:18:36.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.204e-06 2024-09-13T09:18:36.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434205+0000", 2024-09-13T09:18:36.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9689999999999999e-06 2024-09-13T09:18:36.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434207+0000", 2024-09-13T09:18:36.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-13T09:18:36.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490021+0000", 2024-09-13T09:18:36.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055813929999999998 2024-09-13T09:18:36.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490027+0000", 2024-09-13T09:18:36.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.259e-06 2024-09-13T09:18:36.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.847 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.848 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56797 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.848 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:36.848 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.326972644, 2024-09-13T09:18:36.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055843649000000002, 2024-09-13T09:18:36.849 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.849 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.849 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.849 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.849 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.850 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56797 2024-09-13T09:18:36.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.850 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:36.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:36.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:36.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434210+0000", 2024-09-13T09:18:36.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2389999999999999e-06 2024-09-13T09:18:36.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434213+0000", 2024-09-13T09:18:36.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:18:36.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434216+0000", 2024-09-13T09:18:36.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7800000000000001e-06 2024-09-13T09:18:36.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490048+0000", 2024-09-13T09:18:36.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055831981000000003 2024-09-13T09:18:36.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490053+0000", 2024-09-13T09:18:36.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8640000000000004e-06 2024-09-13T09:18:36.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.857 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.858 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56807 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.858 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:36.858 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3252924859999999, 2024-09-13T09:18:36.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054407720999999999, 2024-09-13T09:18:36.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56807 2024-09-13T09:18:36.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.860 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:36.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:36.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435889+0000", 2024-09-13T09:18:36.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435891+0000", 2024-09-13T09:18:36.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.217e-06 2024-09-13T09:18:36.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435894+0000", 2024-09-13T09:18:36.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.067e-06 2024-09-13T09:18:36.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435896+0000", 2024-09-13T09:18:36.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-13T09:18:36.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490292+0000", 2024-09-13T09:18:36.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054395840000000001 2024-09-13T09:18:36.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490297+0000", 2024-09-13T09:18:36.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:18:36.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.868 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.869 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56808 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.869 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:36.869 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.32528366, 2024-09-13T09:18:36.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05442524, 2024-09-13T09:18:36.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56808 2024-09-13T09:18:36.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.871 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:36.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:36.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:36.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435900+0000", 2024-09-13T09:18:36.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:18:36.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435903+0000", 2024-09-13T09:18:36.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:18:36.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435905+0000", 2024-09-13T09:18:36.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:18:36.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490319+0000", 2024-09-13T09:18:36.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054413396000000003 2024-09-13T09:18:36.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490324+0000", 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56809 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3252736039999999, 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054442786, 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56809 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:36.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435910+0000", 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435913+0000", 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1470000000000001e-06 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435915+0000", 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6769999999999999e-06 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490346+0000", 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054430574000000002 2024-09-13T09:18:36.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490351+0000", 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2700000000000004e-06 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56810 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3252643049999999, 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054459855000000001, 2024-09-13T09:18:36.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56810 2024-09-13T09:18:36.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.883 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:36.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435917+0000", 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435919+0000", 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.457e-06 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435924+0000", 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435929+0000", 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8300000000000003e-06 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490373+0000", 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054443841999999999 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490378+0000", 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8509999999999998e-06 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.886 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56811 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3252352039999999, 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054456683999999998, 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56811 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:36.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435946+0000", 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435949+0000", 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435956+0000", 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8719999999999996e-06 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435962+0000", 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000004e-06 2024-09-13T09:18:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490398+0000", 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054436824000000002 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490403+0000", 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0710000000000002e-06 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:36.891 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:36.891 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:36.893 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:36.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:36.893 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:36.890+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:36.894 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:36.921 INFO:teuthology.orchestra.run.smithi032.stdout:3815 writesame 0x954690c thru 0x9555309 data_size 0x2(0xe9fe bytes) 2024-09-13T09:18:36.921 INFO:teuthology.orchestra.run.smithi032.stdout:3816 punch from 0x2c9f6a4 to 0x2ca802a, (0x8986 bytes) 2024-09-13T09:18:36.922 INFO:teuthology.orchestra.run.smithi032.stdout:3818 write 0x5579b83 thru 0x557d2cf (0x374d bytes) 2024-09-13T09:18:36.955 INFO:teuthology.orchestra.run.smithi032.stdout:3820 write 0xa61501f thru 0xa6170ee (0x20d0 bytes) 2024-09-13T09:18:37.031 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:36.990+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:37.033 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:37.033 INFO:teuthology.orchestra.run.smithi032.stdout:3822 write 0x8ff5282 thru 0x8ffc795 (0x7514 bytes) 2024-09-13T09:18:37.048 INFO:teuthology.orchestra.run.smithi032.stdout:3823 write 0x80fb749 thru 0x80fc94d (0x1205 bytes) 2024-09-13T09:18:37.080 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:37.080 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:37.081 INFO:teuthology.orchestra.run.smithi032.stdout:3824 write 0xcaf46f3 thru 0xcb011f5 (0xcb03 bytes) 2024-09-13T09:18:37.093 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:37.090+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:37.098 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:37.105 INFO:teuthology.orchestra.run.smithi032.stdout:3826 trunc from 0xcb011f6 to 0xabb0c1d 2024-09-13T09:18:37.124 INFO:teuthology.orchestra.run.smithi032.stdout:3828 write 0x85ee479 thru 0x85f22b4 (0x3e3c bytes) 2024-09-13T09:18:37.134 INFO:teuthology.orchestra.run.smithi032.stdout:3829 write 0x5278a31 thru 0x527be47 (0x3417 bytes) 2024-09-13T09:18:37.144 INFO:teuthology.orchestra.run.smithi032.stdout:3830 write 0x1cba428 thru 0x1cc1c09 (0x77e2 bytes) 2024-09-13T09:18:37.153 INFO:teuthology.orchestra.run.smithi032.stdout:3831 read 0x3111ad4 thru 0x3121069 (0xf596 bytes) 2024-09-13T09:18:37.154 INFO:teuthology.orchestra.run.smithi032.stdout:3832 write 0x2cd8fb9 thru 0x2cdbead (0x2ef5 bytes) 2024-09-13T09:18:37.178 INFO:teuthology.orchestra.run.smithi032.stdout:3833 write 0xc8b308d thru 0xc8bdd76 (0xacea bytes) 2024-09-13T09:18:37.188 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:37.188 INFO:teuthology.orchestra.run.smithi032.stdout:3837 read 0x1b0e3b thru 0x1bdecb (0xd091 bytes) 2024-09-13T09:18:37.192 INFO:teuthology.orchestra.run.smithi032.stdout:3838 writesame 0x159446d thru 0x1595ccb data_size 0x1(0x185f bytes) 2024-09-13T09:18:37.194 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:37.190+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:37.202 INFO:teuthology.orchestra.run.smithi032.stdout:3840 write 0x389a3ce thru 0x38a5b0a (0xb73d bytes) 2024-09-13T09:18:37.212 INFO:teuthology.orchestra.run.smithi032.stdout:3842 punch from 0xc7521b8 to 0xc75d6b2, (0xb4fa bytes) 2024-09-13T09:18:37.212 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:37.212 INFO:teuthology.orchestra.run.smithi032.stdout:3845 punch from 0xaad0975 to 0xaad306c, (0x26f7 bytes) 2024-09-13T09:18:37.213 INFO:teuthology.orchestra.run.smithi032.stdout:3846 writesame 0x7d60bbb thru 0x7d70075 data_size 0x2b(0xf4bb bytes) 2024-09-13T09:18:37.222 INFO:teuthology.orchestra.run.smithi032.stdout:3847 writesame 0xb89b14f thru 0xb8a0621 data_size 0x5(0x54d3 bytes) 2024-09-13T09:18:37.231 INFO:teuthology.orchestra.run.smithi032.stdout:3848 read 0x256183a thru 0x2564db9 (0x3580 bytes) 2024-09-13T09:18:37.231 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:37.231 INFO:teuthology.orchestra.run.smithi032.stdout:3850 write 0x9fc9c2c thru 0x9fd06f9 (0x6ace bytes) 2024-09-13T09:18:37.241 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:37.241 INFO:teuthology.orchestra.run.smithi032.stdout:3853 write 0xbd2835 thru 0xbd542d (0x2bf9 bytes) 2024-09-13T09:18:37.253 INFO:teuthology.orchestra.run.smithi032.stdout:3854 writesame 0xa1826b0 thru 0xa189eaa data_size 0x1(0x77fb bytes) 2024-09-13T09:18:37.255 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:37.255 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:37.273 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:37.274 INFO:teuthology.orchestra.run.smithi032.stdout:3855 punch from 0xa5407c0 to 0xa545a57, (0x5297 bytes) 2024-09-13T09:18:37.274 INFO:teuthology.orchestra.run.smithi032.stdout:3857 punch from 0xc5afbac to 0xc5b15fe, (0x1a52 bytes) 2024-09-13T09:18:37.274 INFO:teuthology.orchestra.run.smithi032.stdout:3858 read 0x4831326 thru 0x483d3d3 (0xc0ae bytes) 2024-09-13T09:18:37.275 INFO:teuthology.orchestra.run.smithi032.stdout:3861 punch from 0x105c2f3 to 0x106272c, (0x6439 bytes) 2024-09-13T09:18:37.275 INFO:teuthology.orchestra.run.smithi032.stdout:3862 read 0x7d340b6 thru 0x7d429d3 (0xe91e bytes) 2024-09-13T09:18:37.275 INFO:teuthology.orchestra.run.smithi032.stdout:3864 write 0x15c91e4 thru 0x15d0e0f (0x7c2c bytes) 2024-09-13T09:18:37.280 INFO:teuthology.orchestra.run.smithi032.stdout:3865 read 0x10f7ee7 thru 0x10ffc63 (0x7d7d bytes) 2024-09-13T09:18:37.281 INFO:teuthology.orchestra.run.smithi032.stdout:3866 read 0x826b433 thru 0x82780c7 (0xcc95 bytes) 2024-09-13T09:18:37.297 INFO:teuthology.orchestra.run.smithi032.stdout:3867 trunc from 0xc8bdd77 to 0x6522804 2024-09-13T09:18:37.317 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:37.314+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:37.322 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:18:37.322 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:18:37.322 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:18:37.333 INFO:teuthology.orchestra.run.smithi032.stdout:3868 write 0x4599d2e thru 0x45a250c (0x87df bytes) 2024-09-13T09:18:37.357 INFO:teuthology.orchestra.run.smithi032.stdout:3870 writesame 0x9d7fc50 thru 0x9d85b06 data_size 0x1(0x5eb7 bytes) 2024-09-13T09:18:37.358 INFO:teuthology.orchestra.run.smithi032.stdout:3871 writesame 0x9640be3 thru 0x9645bc6 data_size 0x2(0x4fe4 bytes) 2024-09-13T09:18:37.386 INFO:teuthology.orchestra.run.smithi032.stdout:3872 trunc from 0x9d85b07 to 0x2d33617 2024-09-13T09:18:37.396 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:37.394+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:37.431 INFO:teuthology.orchestra.run.smithi032.stdout:3873 trunc from 0x2d33617 to 0xdef6556 2024-09-13T09:18:37.440 INFO:teuthology.orchestra.run.smithi032.stdout:3875 writesame 0xd537875 thru 0xd543c1e data_size 0x5(0xc3aa bytes) 2024-09-13T09:18:37.449 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:37.449 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:3876 read 0x2c26b0 thru 0x2d153b (0xee8c bytes) 2024-09-13T09:18:37.461 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:37.462 INFO:teuthology.orchestra.run.smithi032.stdout:3880 read 0x74da357 thru 0x74de579 (0x4223 bytes) 2024-09-13T09:18:37.462 INFO:teuthology.orchestra.run.smithi032.stdout:3885 punch from 0xa1fbfea to 0xa1fd854, (0x186a bytes) 2024-09-13T09:18:37.462 INFO:teuthology.orchestra.run.smithi032.stdout:3886 punch from 0x2205873 to 0x220a534, (0x4cc1 bytes) 2024-09-13T09:18:37.462 INFO:teuthology.orchestra.run.smithi032.stdout:3887 read 0x57abf2b thru 0x57b6811 (0xa8e7 bytes) 2024-09-13T09:18:37.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:37.473 INFO:teuthology.orchestra.run.smithi032.stdout:3888 trunc from 0xdef6556 to 0x6e8a969 2024-09-13T09:18:37.492 INFO:teuthology.orchestra.run.smithi032.stdout:3889 trunc from 0x6e8a969 to 0x8b81f32 2024-09-13T09:18:37.497 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:37.494+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:37.498 INFO:teuthology.orchestra.run.smithi032.stdout:3890 writesame 0x5e4178d thru 0x5e4bd98 data_size 0x2(0xa60c bytes) 2024-09-13T09:18:37.515 INFO:teuthology.orchestra.run.smithi032.stdout:3891 punch from 0x484a4d4 to 0x4854ee8, (0xaa14 bytes) 2024-09-13T09:18:37.516 INFO:teuthology.orchestra.run.smithi032.stdout:3893 read 0x1af739d thru 0x1afeb4e (0x77b2 bytes) 2024-09-13T09:18:37.518 INFO:teuthology.orchestra.run.smithi032.stdout:3895 trunc from 0x8b81f32 to 0xabeb8ea 2024-09-13T09:18:37.526 INFO:teuthology.orchestra.run.smithi032.stdout:3896 writesame 0x1ff7fb7 thru 0x200533e data_size 0x1c(0xd388 bytes) 2024-09-13T09:18:37.556 INFO:teuthology.orchestra.run.smithi032.stdout:3897 writesame 0xce2b521 thru 0xce3b12f data_size 0x3(0xfc0f bytes) 2024-09-13T09:18:37.556 INFO:teuthology.orchestra.run.smithi032.stdout:3898 read 0x47bca62 thru 0x47c086f (0x3e0e bytes) 2024-09-13T09:18:37.566 INFO:teuthology.orchestra.run.smithi032.stdout:3899 write 0xe0b7fc5 thru 0xe0c2a97 (0xaad3 bytes) 2024-09-13T09:18:37.585 INFO:teuthology.orchestra.run.smithi032.stdout:3900 read 0x1cee29b thru 0x1cf798b (0x96f1 bytes) 2024-09-13T09:18:37.586 INFO:teuthology.orchestra.run.smithi032.stdout:3901 read 0x138e630 thru 0x139d28b (0xec5c bytes) 2024-09-13T09:18:37.587 INFO:teuthology.orchestra.run.smithi032.stdout:3903 punch from 0x4b15783 to 0x4b20de7, (0xb664 bytes) 2024-09-13T09:18:37.587 INFO:teuthology.orchestra.run.smithi032.stdout:3904 read 0x21af67f thru 0x21bcf08 (0xd88a bytes) 2024-09-13T09:18:37.587 INFO:teuthology.orchestra.run.smithi032.stdout:3905 read 0xbfd5ad6 thru 0xbfe443b (0xe966 bytes) 2024-09-13T09:18:37.587 INFO:teuthology.orchestra.run.smithi032.stdout:3906 write 0xb267c47 thru 0xb2696ee (0x1aa8 bytes) 2024-09-13T09:18:37.598 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:37.594+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:37.600 INFO:teuthology.orchestra.run.smithi032.stdout:3908 write 0xb722399 thru 0xb729fd5 (0x7c3d bytes) 2024-09-13T09:18:37.606 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:37.606 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:37.615 INFO:teuthology.orchestra.run.smithi032.stdout:3909 read 0x1a8ce3 thru 0x1ab36a (0x2688 bytes) 2024-09-13T09:18:37.616 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:37.616 INFO:teuthology.orchestra.run.smithi032.stdout:3911 read 0x40642ad thru 0x4065178 (0xecc bytes) 2024-09-13T09:18:37.617 INFO:teuthology.orchestra.run.smithi032.stdout:3912 writesame 0x7dd0ad3 thru 0x7dd69d5 data_size 0x1(0x5f03 bytes) 2024-09-13T09:18:37.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:37.640 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:37.640 INFO:teuthology.orchestra.run.smithi032.stdout:3914 write 0x8827b9f thru 0x882eccd (0x712f bytes) 2024-09-13T09:18:37.653 INFO:teuthology.orchestra.run.smithi032.stdout:3916 read 0xdf723e2 thru 0xdf82351 (0xff70 bytes) 2024-09-13T09:18:37.653 INFO:teuthology.orchestra.run.smithi032.stdout:3917 writesame 0x41ccb48 thru 0x41d5392 data_size 0x25(0x884b bytes) 2024-09-13T09:18:37.669 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:18:37.670 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:18:37.683 INFO:teuthology.orchestra.run.smithi032.stdout:3918 trunc from 0xe0c2a98 to 0x31d93bf 2024-09-13T09:18:37.699 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:37.698+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:37.700 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-13T09:18:36.201076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":862115169,"num_objects":963,"num_object_clones":249,"num_object_copies":1926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":963,"num_whiteouts":270,"num_read":75324,"num_read_kb":281422,"num_write":12246,"num_write_kb":194987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":780,"num_bytes_recovered":10426586,"num_keys_recovered":157,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9359,"ondisk_log_size":9359,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":350580,"kb_used_data":40972,"kb_used_omap":12,"kb_used_meta":309491,"kb_avail":754624140,"statfs":{"total":773094113280,"available":772735119360,"internally_reserved":0,"allocated":41955328,"data_stored":256177212,"data_compressed":6286111,"data_compressed_allocated":17461248,"data_compressed_original":238357087,"omap_allocated":12744,"internal_metadata":316919352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":126043780,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":2497,"num_read_kb":29779,"num_write":508,"num_write_kb":17285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":4424839,"num_keys_recovered":30,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.120108"},"pg_stats":[{"pgid":"3.1f","version":"77'229","reported_seq":1254,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.921224+0000","last_change":"2024-09-13T09:17:40.849739+0000","last_active":"2024-09-13T09:18:32.921224+0000","last_peered":"2024-09-13T09:18:32.921224+0000","last_clean":"2024-09-13T09:18:32.921224+0000","last_became_active":"2024-09-13T09:17:40.791187+0000","last_became_peered":"2024-09-13T09:17:40.791187+0000","last_unstale":"2024-09-13T09:18:32.921224+0000","last_undegraded":"2024-09-13T09:18:32.921224+0000","last_fullsized":"2024-09-13T09:18:32.921224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:13.280600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046778999999999999,"stat_sum":{"num_bytes":37045926,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":7,"num_read":1867,"num_read_kb":12197,"num_write":298,"num_write_kb":10313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"78'517","reported_seq":2451,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.844632+0000","last_change":"2024-09-13T09:18:32.182614+0000","last_active":"2024-09-13T09:18:33.844632+0000","last_peered":"2024-09-13T09:18:33.844632+0000","last_clean":"2024-09-13T09:18:33.844632+0000","last_became_active":"2024-09-13T09:18:32.181112+0000","last_became_peered":"2024-09-13T09:18:32.181112+0000","last_unstale":"2024-09-13T09:18:33.844632+0000","last_undegraded":"2024-09-13T09:18:33.844632+0000","last_fullsized":"2024-09-13T09:18:33.844632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:07.863655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382264,"stat_sum":{"num_bytes":22848233,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":15,"num_read":3501,"num_read_kb":9610,"num_write":771,"num_write_kb":4002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":2908199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"69'215","reported_seq":2053,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.196167+0000","last_change":"2024-09-13T09:17:28.692127+0000","last_active":"2024-09-13T09:18:32.196167+0000","last_peered":"2024-09-13T09:18:32.196167+0000","last_clean":"2024-09-13T09:18:32.196167+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:18:32.196167+0000","last_undegraded":"2024-09-13T09:18:32.196167+0000","last_fullsized":"2024-09-13T09:18:32.196167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:20.509991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001448682,"stat_sum":{"num_bytes":25302911,"num_objects":32,"num_object_clones":11,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":3076,"num_read_kb":8541,"num_write":241,"num_write_kb":9559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"77'401","reported_seq":2204,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.397714+0000","last_change":"2024-09-13T09:17:28.691986+0000","last_active":"2024-09-13T09:18:32.397714+0000","last_peered":"2024-09-13T09:18:32.397714+0000","last_clean":"2024-09-13T09:18:32.397714+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:18:32.397714+0000","last_undegraded":"2024-09-13T09:18:32.397714+0000","last_fullsized":"2024-09-13T09:18:32.397714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:05.002231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013280939999999999,"stat_sum":{"num_bytes":5459884,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":8,"num_read":4590,"num_read_kb":6820,"num_write":472,"num_write_kb":2333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"78'403","reported_seq":3066,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.750855+0000","last_change":"2024-09-13T09:17:40.897770+0000","last_active":"2024-09-13T09:18:33.750855+0000","last_peered":"2024-09-13T09:18:33.750855+0000","last_clean":"2024-09-13T09:18:33.750855+0000","last_became_active":"2024-09-13T09:17:40.799618+0000","last_became_peered":"2024-09-13T09:17:40.799618+0000","last_unstale":"2024-09-13T09:18:33.750855+0000","last_undegraded":"2024-09-13T09:18:33.750855+0000","last_fullsized":"2024-09-13T09:18:33.750855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:00.757881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042467999999999998,"stat_sum":{"num_bytes":35505835,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":4500,"num_read_kb":10370,"num_write":551,"num_write_kb":6099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":65341,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"76'154","reported_seq":1813,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.785633+0000","last_change":"2024-09-13T09:18:04.085791+0000","last_active":"2024-09-13T09:18:33.785633+0000","last_peered":"2024-09-13T09:18:33.785633+0000","last_clean":"2024-09-13T09:18:33.785633+0000","last_became_active":"2024-09-13T09:18:03.973063+0000","last_became_peered":"2024-09-13T09:18:03.973063+0000","last_unstale":"2024-09-13T09:18:33.785633+0000","last_undegraded":"2024-09-13T09:18:33.785633+0000","last_fullsized":"2024-09-13T09:18:33.785633+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:29.911994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077588799999999997,"stat_sum":{"num_bytes":12844288,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":1793,"num_read_kb":4616,"num_write":150,"num_write_kb":3143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":889520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"78'562","reported_seq":1975,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.744703+0000","last_change":"2024-09-13T09:18:06.977335+0000","last_active":"2024-09-13T09:18:33.744703+0000","last_peered":"2024-09-13T09:18:33.744703+0000","last_clean":"2024-09-13T09:18:33.744703+0000","last_became_active":"2024-09-13T09:18:06.975016+0000","last_became_peered":"2024-09-13T09:18:06.975016+0000","last_unstale":"2024-09-13T09:18:33.744703+0000","last_undegraded":"2024-09-13T09:18:33.744703+0000","last_fullsized":"2024-09-13T09:18:33.744703+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:30.489567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023026689999999998,"stat_sum":{"num_bytes":52278349,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":9,"num_read":3526,"num_read_kb":10401,"num_write":891,"num_write_kb":9664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":439303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"76'122","reported_seq":1533,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.174828+0000","last_change":"2024-09-13T09:17:40.836458+0000","last_active":"2024-09-13T09:18:32.174828+0000","last_peered":"2024-09-13T09:18:32.174828+0000","last_clean":"2024-09-13T09:18:32.174828+0000","last_became_active":"2024-09-13T09:17:40.797607+0000","last_became_peered":"2024-09-13T09:17:40.797607+0000","last_unstale":"2024-09-13T09:18:32.174828+0000","last_undegraded":"2024-09-13T09:18:32.174828+0000","last_fullsized":"2024-09-13T09:18:32.174828+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:09.025976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001136335,"stat_sum":{"num_bytes":32838420,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":1351,"num_read_kb":7011,"num_write":114,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":42856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"76'206","reported_seq":1294,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.687705+0000","last_change":"2024-09-13T09:17:43.811230+0000","last_active":"2024-09-13T09:18:32.687705+0000","last_peered":"2024-09-13T09:18:32.687705+0000","last_clean":"2024-09-13T09:18:32.687705+0000","last_became_active":"2024-09-13T09:17:43.809533+0000","last_became_peered":"2024-09-13T09:17:43.809533+0000","last_unstale":"2024-09-13T09:18:32.687705+0000","last_undegraded":"2024-09-13T09:18:32.687705+0000","last_fullsized":"2024-09-13T09:18:32.687705+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:20.389195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001841894,"stat_sum":{"num_bytes":15359762,"num_objects":34,"num_object_clones":12,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":1825,"num_read_kb":16337,"num_write":229,"num_write_kb":8685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":16384,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"78'158","reported_seq":1165,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.751535+0000","last_change":"2024-09-13T09:17:28.692331+0000","last_active":"2024-09-13T09:18:33.751535+0000","last_peered":"2024-09-13T09:18:33.751535+0000","last_clean":"2024-09-13T09:18:33.751535+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:18:33.751535+0000","last_undegraded":"2024-09-13T09:18:33.751535+0000","last_fullsized":"2024-09-13T09:18:33.751535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:29.849557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016477340000000001,"stat_sum":{"num_bytes":35853456,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":1089,"num_read_kb":6297,"num_write":166,"num_write_kb":1529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"78'161","reported_seq":988,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.831625+0000","last_change":"2024-09-13T09:17:28.684144+0000","last_active":"2024-09-13T09:18:33.831625+0000","last_peered":"2024-09-13T09:18:33.831625+0000","last_clean":"2024-09-13T09:18:33.831625+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:18:33.831625+0000","last_undegraded":"2024-09-13T09:18:33.831625+0000","last_fullsized":"2024-09-13T09:18:33.831625+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:10.717628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085639099999999997,"stat_sum":{"num_bytes":25560410,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":821,"num_read_kb":8257,"num_write":239,"num_write_kb":12403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"78'425","reported_seq":1010,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.739359+0000","last_change":"2024-09-13T09:17:28.681431+0000","last_active":"2024-09-13T09:18:33.739359+0000","last_peered":"2024-09-13T09:18:33.739359+0000","last_clean":"2024-09-13T09:18:33.739359+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:18:33.739359+0000","last_undegraded":"2024-09-13T09:18:33.739359+0000","last_fullsized":"2024-09-13T09:18:33.739359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:42.341412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083138199999999998,"stat_sum":{"num_bytes":55710480,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":5,"num_read":1630,"num_read_kb":6544,"num_write":624,"num_write_kb":12338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.178842+0000","last_change":"2024-09-13T09:18:28.590006+0000","last_active":"2024-09-13T09:18:32.178842+0000","last_peered":"2024-09-13T09:18:32.178842+0000","last_clean":"2024-09-13T09:18:32.178842+0000","last_became_active":"2024-09-13T09:18:28.589015+0000","last_became_peered":"2024-09-13T09:18:28.589015+0000","last_unstale":"2024-09-13T09:18:32.178842+0000","last_undegraded":"2024-09-13T09:18:32.178842+0000","last_fullsized":"2024-09-13T09:18:32.178842+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:31:41.738463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"74'220","reported_seq":1000,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.434892+0000","last_change":"2024-09-13T09:18:04.241645+0000","last_active":"2024-09-13T09:18:32.434892+0000","last_peered":"2024-09-13T09:18:32.434892+0000","last_clean":"2024-09-13T09:18:32.434892+0000","last_became_active":"2024-09-13T09:18:03.979234+0000","last_became_peered":"2024-09-13T09:18:03.979234+0000","last_unstale":"2024-09-13T09:18:32.434892+0000","last_undegraded":"2024-09-13T09:18:32.434892+0000","last_fullsized":"2024-09-13T09:18:32.434892+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:42:02.026772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002893765,"stat_sum":{"num_bytes":45833439,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":10,"num_read":1988,"num_read_kb":16233,"num_write":233,"num_write_kb":9543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":380627,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.282266+0000","last_change":"2024-09-13T09:18:28.151410+0000","last_active":"2024-09-13T09:18:32.282266+0000","last_peered":"2024-09-13T09:18:32.282266+0000","last_clean":"2024-09-13T09:18:32.282266+0000","last_became_active":"2024-09-13T09:18:28.150686+0000","last_became_peered":"2024-09-13T09:18:28.150686+0000","last_unstale":"2024-09-13T09:18:32.282266+0000","last_undegraded":"2024-09-13T09:18:32.282266+0000","last_fullsized":"2024-09-13T09:18:32.282266+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:53.354314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"76'393","reported_seq":1647,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.175426+0000","last_change":"2024-09-13T09:17:28.684573+0000","last_active":"2024-09-13T09:18:32.175426+0000","last_peered":"2024-09-13T09:18:32.175426+0000","last_clean":"2024-09-13T09:18:32.175426+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:18:32.175426+0000","last_undegraded":"2024-09-13T09:18:32.175426+0000","last_fullsized":"2024-09-13T09:18:32.175426+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:51.225828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086084099999999999,"stat_sum":{"num_bytes":12342952,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":2577,"num_read_kb":4534,"num_write":585,"num_write_kb":1519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":88,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.172583+0000","last_change":"2024-09-13T09:17:40.794662+0000","last_active":"2024-09-13T09:18:32.172583+0000","last_peered":"2024-09-13T09:18:32.172583+0000","last_clean":"2024-09-13T09:18:32.172583+0000","last_became_active":"2024-09-13T09:17:40.793882+0000","last_became_peered":"2024-09-13T09:17:40.793882+0000","last_unstale":"2024-09-13T09:18:32.172583+0000","last_undegraded":"2024-09-13T09:18:32.172583+0000","last_fullsized":"2024-09-13T09:18:32.172583+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:22.922753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"77'159","reported_seq":902,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.693210+0000","last_change":"2024-09-13T09:18:04.163142+0000","last_active":"2024-09-13T09:18:32.693210+0000","last_peered":"2024-09-13T09:18:32.693210+0000","last_clean":"2024-09-13T09:18:32.693210+0000","last_became_active":"2024-09-13T09:18:03.973314+0000","last_became_peered":"2024-09-13T09:18:03.973314+0000","last_unstale":"2024-09-13T09:18:32.693210+0000","last_undegraded":"2024-09-13T09:18:32.693210+0000","last_fullsized":"2024-09-13T09:18:32.693210+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.256457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024125790000000002,"stat_sum":{"num_bytes":34193109,"num_objects":40,"num_object_clones":14,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":15,"num_read":972,"num_read_kb":11667,"num_write":153,"num_write_kb":8030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":167476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.281939+0000","last_change":"2024-09-13T09:18:03.968679+0000","last_active":"2024-09-13T09:18:32.281939+0000","last_peered":"2024-09-13T09:18:32.281939+0000","last_clean":"2024-09-13T09:18:32.281939+0000","last_became_active":"2024-09-13T09:18:03.967133+0000","last_became_peered":"2024-09-13T09:18:03.967133+0000","last_unstale":"2024-09-13T09:18:32.281939+0000","last_undegraded":"2024-09-13T09:18:32.281939+0000","last_fullsized":"2024-09-13T09:18:32.281939+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:06.643934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"77'217","reported_seq":1839,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.688053+0000","last_change":"2024-09-13T09:17:28.692726+0000","last_active":"2024-09-13T09:18:32.688053+0000","last_peered":"2024-09-13T09:18:32.688053+0000","last_clean":"2024-09-13T09:18:32.688053+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:18:32.688053+0000","last_undegraded":"2024-09-13T09:18:32.688053+0000","last_fullsized":"2024-09-13T09:18:32.688053+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:00.161632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018814960000000001,"stat_sum":{"num_bytes":50199363,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":10,"num_read":2030,"num_read_kb":7746,"num_write":260,"num_write_kb":3508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"78'191","reported_seq":2069,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.640656+0000","last_change":"2024-09-13T09:17:40.875161+0000","last_active":"2024-09-13T09:18:33.640656+0000","last_peered":"2024-09-13T09:18:33.640656+0000","last_clean":"2024-09-13T09:18:33.640656+0000","last_became_active":"2024-09-13T09:17:40.800027+0000","last_became_peered":"2024-09-13T09:17:40.800027+0000","last_unstale":"2024-09-13T09:18:33.640656+0000","last_undegraded":"2024-09-13T09:18:33.640656+0000","last_fullsized":"2024-09-13T09:18:33.640656+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:10.182783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066593499999999999,"stat_sum":{"num_bytes":13506017,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":2194,"num_read_kb":6245,"num_write":239,"num_write_kb":819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":29248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":110,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.196284+0000","last_change":"2024-09-13T09:18:03.976890+0000","last_active":"2024-09-13T09:18:32.196284+0000","last_peered":"2024-09-13T09:18:32.196284+0000","last_clean":"2024-09-13T09:18:32.196284+0000","last_became_active":"2024-09-13T09:18:03.970835+0000","last_became_peered":"2024-09-13T09:18:03.970835+0000","last_unstale":"2024-09-13T09:18:32.196284+0000","last_undegraded":"2024-09-13T09:18:32.196284+0000","last_fullsized":"2024-09-13T09:18:32.196284+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:05.189083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"76'256","reported_seq":1155,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.281843+0000","last_change":"2024-09-13T09:18:04.075705+0000","last_active":"2024-09-13T09:18:32.281843+0000","last_peered":"2024-09-13T09:18:32.281843+0000","last_clean":"2024-09-13T09:18:32.281843+0000","last_became_active":"2024-09-13T09:18:03.970479+0000","last_became_peered":"2024-09-13T09:18:03.970479+0000","last_unstale":"2024-09-13T09:18:32.281843+0000","last_undegraded":"2024-09-13T09:18:32.281843+0000","last_fullsized":"2024-09-13T09:18:32.281843+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:06.229840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043683300000000002,"stat_sum":{"num_bytes":11173670,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":8,"num_read":3171,"num_read_kb":6030,"num_write":290,"num_write_kb":3339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1031390,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.282318+0000","last_change":"2024-09-13T09:17:19.575553+0000","last_active":"2024-09-13T09:18:32.282318+0000","last_peered":"2024-09-13T09:18:32.282318+0000","last_clean":"2024-09-13T09:18:32.282318+0000","last_became_active":"2024-09-13T09:17:17.493930+0000","last_became_peered":"2024-09-13T09:17:17.493930+0000","last_unstale":"2024-09-13T09:18:32.282318+0000","last_undegraded":"2024-09-13T09:18:32.282318+0000","last_fullsized":"2024-09-13T09:18:32.282318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:12.092799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000590198,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"77'161","reported_seq":1399,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.686617+0000","last_change":"2024-09-13T09:17:40.870470+0000","last_active":"2024-09-13T09:18:32.686617+0000","last_peered":"2024-09-13T09:18:32.686617+0000","last_clean":"2024-09-13T09:18:32.686617+0000","last_became_active":"2024-09-13T09:17:40.793065+0000","last_became_peered":"2024-09-13T09:17:40.793065+0000","last_unstale":"2024-09-13T09:18:32.686617+0000","last_undegraded":"2024-09-13T09:18:32.686617+0000","last_fullsized":"2024-09-13T09:18:32.686617+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:38.586460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052255699999999999,"stat_sum":{"num_bytes":25112769,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":9,"num_read":1267,"num_read_kb":15398,"num_write":160,"num_write_kb":6911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.172129+0000","last_change":"2024-09-13T09:18:03.968508+0000","last_active":"2024-09-13T09:18:32.172129+0000","last_peered":"2024-09-13T09:18:32.172129+0000","last_clean":"2024-09-13T09:18:32.172129+0000","last_became_active":"2024-09-13T09:18:03.968092+0000","last_became_peered":"2024-09-13T09:18:03.968092+0000","last_unstale":"2024-09-13T09:18:32.172129+0000","last_undegraded":"2024-09-13T09:18:32.172129+0000","last_fullsized":"2024-09-13T09:18:32.172129+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:39.649827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"78'199","reported_seq":1400,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.883450+0000","last_change":"2024-09-13T09:17:28.686727+0000","last_active":"2024-09-13T09:18:33.883450+0000","last_peered":"2024-09-13T09:18:33.883450+0000","last_clean":"2024-09-13T09:18:33.883450+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:18:33.883450+0000","last_undegraded":"2024-09-13T09:18:33.883450+0000","last_fullsized":"2024-09-13T09:18:33.883450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:44.094850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061431400000000003,"stat_sum":{"num_bytes":41429745,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":1646,"num_read_kb":8435,"num_write":248,"num_write_kb":9077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":73,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.196061+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:18:32.196061+0000","last_peered":"2024-09-13T09:18:32.196061+0000","last_clean":"2024-09-13T09:18:32.196061+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:18:32.196061+0000","last_undegraded":"2024-09-13T09:18:32.196061+0000","last_fullsized":"2024-09-13T09:18:32.196061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":263,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.172232+0000","last_change":"2024-09-13T09:17:43.803525+0000","last_active":"2024-09-13T09:18:32.172232+0000","last_peered":"2024-09-13T09:18:32.172232+0000","last_clean":"2024-09-13T09:18:32.172232+0000","last_became_active":"2024-09-13T09:17:43.802603+0000","last_became_peered":"2024-09-13T09:17:43.802603+0000","last_unstale":"2024-09-13T09:18:32.172232+0000","last_undegraded":"2024-09-13T09:18:32.172232+0000","last_fullsized":"2024-09-13T09:18:32.172232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:07.561959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"77'394","reported_seq":1605,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.686462+0000","last_change":"2024-09-13T09:18:04.153662+0000","last_active":"2024-09-13T09:18:32.686462+0000","last_peered":"2024-09-13T09:18:32.686462+0000","last_clean":"2024-09-13T09:18:32.686462+0000","last_became_active":"2024-09-13T09:18:03.974569+0000","last_became_peered":"2024-09-13T09:18:03.974569+0000","last_unstale":"2024-09-13T09:18:32.686462+0000","last_undegraded":"2024-09-13T09:18:32.686462+0000","last_fullsized":"2024-09-13T09:18:32.686462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:43.479751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014243249999999999,"stat_sum":{"num_bytes":36585215,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":11,"num_read":2778,"num_read_kb":8573,"num_write":533,"num_write_kb":3792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":741496,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"74'360","reported_seq":1663,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.384950+0000","last_change":"2024-09-13T09:17:28.681630+0000","last_active":"2024-09-13T09:18:32.384950+0000","last_peered":"2024-09-13T09:18:32.384950+0000","last_clean":"2024-09-13T09:18:32.384950+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:18:32.384950+0000","last_undegraded":"2024-09-13T09:18:32.384950+0000","last_fullsized":"2024-09-13T09:18:32.384950+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:06.071033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013508260000000001,"stat_sum":{"num_bytes":10365911,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":2762,"num_read_kb":7891,"num_write":462,"num_write_kb":2728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"78'273","reported_seq":1701,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.817948+0000","last_change":"2024-09-13T09:18:31.170680+0000","last_active":"2024-09-13T09:18:33.817948+0000","last_peered":"2024-09-13T09:18:33.817948+0000","last_clean":"2024-09-13T09:18:33.817948+0000","last_became_active":"2024-09-13T09:18:31.168524+0000","last_became_peered":"2024-09-13T09:18:31.168524+0000","last_unstale":"2024-09-13T09:18:33.817948+0000","last_undegraded":"2024-09-13T09:18:33.817948+0000","last_fullsized":"2024-09-13T09:18:33.817948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:03:02.797737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033158900000000003,"stat_sum":{"num_bytes":22197256,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":2186,"num_read_kb":6557,"num_write":263,"num_write_kb":3530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1361512,"num_keys_recovered":74,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"76'299","reported_seq":772,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.175180+0000","last_change":"2024-09-13T09:17:40.896216+0000","last_active":"2024-09-13T09:18:32.175180+0000","last_peered":"2024-09-13T09:18:32.175180+0000","last_clean":"2024-09-13T09:18:32.175180+0000","last_became_active":"2024-09-13T09:17:40.801195+0000","last_became_peered":"2024-09-13T09:17:40.801195+0000","last_unstale":"2024-09-13T09:18:32.175180+0000","last_undegraded":"2024-09-13T09:18:32.175180+0000","last_fullsized":"2024-09-13T09:18:32.175180+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:00:07.773428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002522725,"stat_sum":{"num_bytes":19534929,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":1679,"num_read_kb":4080,"num_write":420,"num_write_kb":7756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67680,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"74'363","reported_seq":1027,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.684910+0000","last_change":"2024-09-13T09:17:28.693070+0000","last_active":"2024-09-13T09:18:32.684910+0000","last_peered":"2024-09-13T09:18:32.684910+0000","last_clean":"2024-09-13T09:18:32.684910+0000","last_became_active":"2024-09-13T09:17:26.698657+0000","last_became_peered":"2024-09-13T09:17:26.698657+0000","last_unstale":"2024-09-13T09:18:32.684910+0000","last_undegraded":"2024-09-13T09:18:32.684910+0000","last_fullsized":"2024-09-13T09:18:32.684910+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:12.388714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002213343,"stat_sum":{"num_bytes":13072282,"num_objects":26,"num_object_clones":9,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":1563,"num_read_kb":6161,"num_write":529,"num_write_kb":6452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"76'262","reported_seq":1835,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.172489+0000","last_change":"2024-09-13T09:17:28.684477+0000","last_active":"2024-09-13T09:18:32.172489+0000","last_peered":"2024-09-13T09:18:32.172489+0000","last_clean":"2024-09-13T09:18:32.172489+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:18:32.172489+0000","last_undegraded":"2024-09-13T09:18:32.172489+0000","last_fullsized":"2024-09-13T09:18:32.172489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:25.411265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099816899999999992,"stat_sum":{"num_bytes":37338863,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":12,"num_read":2294,"num_read_kb":12592,"num_write":313,"num_write_kb":13940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"77'238","reported_seq":778,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.352817+0000","last_change":"2024-09-13T09:17:28.687062+0000","last_active":"2024-09-13T09:18:32.352817+0000","last_peered":"2024-09-13T09:18:32.352817+0000","last_clean":"2024-09-13T09:18:32.352817+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:18:32.352817+0000","last_undegraded":"2024-09-13T09:18:32.352817+0000","last_fullsized":"2024-09-13T09:18:32.352817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:31.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034733699999999999,"stat_sum":{"num_bytes":20500716,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":1013,"num_read_kb":3475,"num_write":310,"num_write_kb":2804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"78'839","reported_seq":2936,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.520494+0000","last_change":"2024-09-13T09:17:28.684481+0000","last_active":"2024-09-13T09:18:33.520494+0000","last_peered":"2024-09-13T09:18:33.520494+0000","last_clean":"2024-09-13T09:18:33.520494+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:18:33.520494+0000","last_undegraded":"2024-09-13T09:18:33.520494+0000","last_fullsized":"2024-09-13T09:18:33.520494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":839,"log_dups_size":0,"ondisk_log_size":839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:56.423075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151104,"stat_sum":{"num_bytes":25747294,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":6190,"num_read_kb":11300,"num_write":1325,"num_write_kb":3596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"77'270","reported_seq":1262,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.194844+0000","last_change":"2024-09-13T09:18:31.171539+0000","last_active":"2024-09-13T09:18:33.194844+0000","last_peered":"2024-09-13T09:18:33.194844+0000","last_clean":"2024-09-13T09:18:33.194844+0000","last_became_active":"2024-09-13T09:18:31.167554+0000","last_became_peered":"2024-09-13T09:18:31.167554+0000","last_unstale":"2024-09-13T09:18:33.194844+0000","last_undegraded":"2024-09-13T09:18:33.194844+0000","last_fullsized":"2024-09-13T09:18:33.194844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:55.799591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037557999999999998,"stat_sum":{"num_bytes":8305868,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":8,"num_read":1513,"num_read_kb":6849,"num_write":373,"num_write_kb":4830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":975497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"77'135","reported_seq":1347,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.572461+0000","last_change":"2024-09-13T09:17:28.683788+0000","last_active":"2024-09-13T09:18:32.572461+0000","last_peered":"2024-09-13T09:18:32.572461+0000","last_clean":"2024-09-13T09:18:32.572461+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:18:32.572461+0000","last_undegraded":"2024-09-13T09:18:32.572461+0000","last_fullsized":"2024-09-13T09:18:32.572461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:47.572839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070075400000000005,"stat_sum":{"num_bytes":9718619,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":3,"num_read":1211,"num_read_kb":8419,"num_write":174,"num_write_kb":8291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"78'271","reported_seq":2250,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T09:18:33.746304+0000","last_change":"2024-09-13T09:17:40.816999+0000","last_active":"2024-09-13T09:18:33.746304+0000","last_peered":"2024-09-13T09:18:33.746304+0000","last_clean":"2024-09-13T09:18:33.746304+0000","last_became_active":"2024-09-13T09:17:40.789499+0000","last_became_peered":"2024-09-13T09:17:40.789499+0000","last_unstale":"2024-09-13T09:18:33.746304+0000","last_undegraded":"2024-09-13T09:18:33.746304+0000","last_fullsized":"2024-09-13T09:18:33.746304+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:13.395268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084252600000000002,"stat_sum":{"num_bytes":50682157,"num_objects":35,"num_object_clones":12,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":3244,"num_read_kb":13495,"num_write":277,"num_write_kb":7724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1322,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"77'186","reported_seq":2907,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T09:18:32.369924+0000","last_change":"2024-09-13T09:17:28.683729+0000","last_active":"2024-09-13T09:18:32.369924+0000","last_peered":"2024-09-13T09:18:32.369924+0000","last_clean":"2024-09-13T09:18:32.369924+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:18:32.369924+0000","last_undegraded":"2024-09-13T09:18:32.369924+0000","last_fullsized":"2024-09-13T09:18:32.369924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:07.893848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000594467,"stat_sum":{"num_bytes":17076654,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":9,"num_read":3361,"num_read_kb":8641,"num_write":192,"num_write_kb":2031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":861524782,"num_objects":959,"num_object_clones":249,"num_object_copies":1918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":959,"num_whiteouts":270,"num_read":75208,"num_read_kb":281322,"num_write":12085,"num_write_kb":192569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":9245793,"num_keys_recovered":157,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34627584,"data_stored":251808766,"data_compressed":6266624,"data_compressed_allocated":17395712,"data_compressed_original":237234783,"omap_allocated":0,"internal_metadata":0},"log_size":9239,"ondisk_log_size":9239,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":30095,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":5668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":5804032,"data_stored":35608609,"data_compressed":728824,"data_compressed_allocated":2310144,"data_compressed_original":33050891,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.2360000000000002,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.73199999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.54400000000000004},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":3.9039999999999999,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.71299999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.097},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.75}]},{"osd":4,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":3.6139999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.91300000000000003}]},{"osd":5,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.125,"5min":3.125,"15min":3.125},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.3130000000000002,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.2330000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1904640,"data_stored":4841375,"data_compressed":87748,"data_compressed_allocated":319488,"data_compressed_original":3870720,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":0.67300000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.502},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.018}]},{"osd":2,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.5529999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.751},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.72399999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.84299999999999997}]},{"osd":5,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.872},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.1319999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.98599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46380,"kb_used_data":7068,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94325460,"statfs":{"total":96636764160,"available":96589271040,"internally_reserved":0,"allocated":7237632,"data_stored":49309955,"data_compressed":953225,"data_compressed_allocated":3403776,"data_compressed_original":46420203,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":7.8150000000000004,"5min":7.8150000000000004,"15min":7.8150000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":132.32900000000001,"5min":132.32900000000001,"15min":132.32900000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.75}]},{"osd":1,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.92200000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":8.2629999999999999,"5min":8.2629999999999999,"15min":8.2629999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":132.23599999999999,"5min":132.23599999999999,"15min":132.23599999999999},"last":0.47699999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.442}]},{"osd":7,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.5910000000000002,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":0.53800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55188,"kb_used_data":6024,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94316652,"statfs":{"total":96636764160,"available":96580251648,"internally_reserved":0,"allocated":6168576,"data_stored":36342836,"data_compressed":871233,"data_compressed_allocated":2514944,"data_compressed_original":33621490,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.032},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":1.069}]},{"osd":1,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.79900000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.107},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.1859999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.145},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.98499999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.92700000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.3600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51464,"kb_used_data":6400,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94320376,"statfs":{"total":96636764160,"available":96584065024,"internally_reserved":0,"allocated":6553600,"data_stored":39942820,"data_compressed":821205,"data_compressed_allocated":2658304,"data_compressed_original":37040302,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":1.1459999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.78700000000000003}]},{"osd":4,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.008}]},{"osd":5,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.97499999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.113},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":1.2450000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:18:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.046},{"interface":"front","average":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":15.4,"5min":15.4,"15min":15.4},"last":1.198}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38400,"kb_used_data":3188,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333440,"statfs":{"total":96636764160,"available":96597442560,"internally_reserved":0,"allocated":3264512,"data_stored":21304275,"data_compressed":715283,"data_compressed_allocated":1355776,"data_compressed_original":20018137,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.442},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.755}]},{"osd":3,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.72399999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.81499999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.94299999999999995}]},{"osd":6,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.91700000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:18:11 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.9809999999999999,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.032}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43956,"kb_used_data":4644,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327884,"statfs":{"total":96636764160,"available":96591753216,"internally_reserved":0,"allocated":4755456,"data_stored":30616985,"data_compressed":1011141,"data_compressed_allocated":2125824,"data_compressed_original":28744698,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.255},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.2569999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.052}]},{"osd":3,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":1.006},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.4830000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.099}]},{"osd":5,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.175},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.339},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:18:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.411}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45432,"kb_used_data":6120,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94326408,"statfs":{"total":96636764160,"available":96590241792,"internally_reserved":0,"allocated":6266880,"data_stored":38210357,"data_compressed":1097452,"data_compressed_allocated":2772992,"data_compressed_original":35590646,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.436}]},{"osd":2,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.61499999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.5},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.64400000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.75}]},{"osd":7,"last update":"Fri Sep 13 09:18:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":1}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5312512,"data_stored":37229832,"data_compressed":1087189,"data_compressed_allocated":2740224,"data_compressed_original":35021302,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3833856,"data_stored":30200522,"data_compressed":1011141,"data_compressed_allocated":2125824,"data_compressed_original":28744698,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2347008,"data_stored":20887831,"data_compressed":715283,"data_compressed_allocated":1355776,"data_compressed_original":20018137,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5636096,"data_stored":39526376,"data_compressed":821205,"data_compressed_allocated":2658304,"data_compressed_original":37040302,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5263360,"data_stored":35950111,"data_compressed":871348,"data_compressed_allocated":2519040,"data_compressed_original":33641970,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6365184,"data_stored":48936677,"data_compressed":953225,"data_compressed_allocated":3403776,"data_compressed_original":46420203,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1040384,"data_stored":4475620,"data_compressed":87748,"data_compressed_allocated":319488,"data_compressed_original":3870720,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4829184,"data_stored":34601797,"data_compressed":719485,"data_compressed_allocated":2273280,"data_compressed_original":32477451,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:18:37.709 INFO:teuthology.orchestra.run.smithi032.stdout:3920 punch from 0x2dac825 to 0x2db694e, (0xa129 bytes) 2024-09-13T09:18:37.709 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:37.709 INFO:teuthology.orchestra.run.smithi032.stdout:3922 clone 28 order 25 su 131072 sc 13 2024-09-13T09:18:37.768 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:37.769 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:37.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:37.831 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:37.830+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:37.901 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:37.898+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:37.958 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:37.959 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:37.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:38.023 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:38.022+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:38.101 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:38.101 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:38.102 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:38.098+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:38.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:38.203 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:38.202+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:38.295 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:38.296 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:38.304 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:38.298+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:38.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:38.312 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone27 from 0x31d93bf (overlap 0x9a6cea) to 0x0 2024-09-13T09:18:38.407 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:38.402+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:38.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:18:38.506 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:38.502+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:38.514 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:38.514 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:38.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:38.606 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:38.602+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:38.622 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-13T09:18:38.645 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:38.645 INFO:teuthology.orchestra.run.smithi032.stdout:3924 read 0x1744ae1 thru 0x174b7ed (0x6d0d bytes) 2024-09-13T09:18:38.662 INFO:teuthology.orchestra.run.smithi032.stdout:3925 writesame 0x5068cdb thru 0x506ebae data_size 0x11(0x5ed4 bytes) 2024-09-13T09:18:38.682 INFO:teuthology.orchestra.run.smithi032.stdout:3926 read 0x3091e1c thru 0x3096fd2 (0x51b7 bytes) 2024-09-13T09:18:38.683 INFO:teuthology.orchestra.run.smithi032.stdout:3927 read 0x2909b4b thru 0x290f9ba (0x5e70 bytes) 2024-09-13T09:18:38.683 INFO:teuthology.orchestra.run.smithi032.stdout:3928 writesame 0x16d3dec thru 0x16da1b7 data_size 0x6(0x63cc bytes) 2024-09-13T09:18:38.699 INFO:teuthology.orchestra.run.smithi032.stdout:3930 read 0x2988b7d thru 0x299495d (0xbde1 bytes) 2024-09-13T09:18:38.707 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:38.702+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:38.711 INFO:teuthology.orchestra.run.smithi032.stdout:3931 trunc from 0x506ebaf to 0xc38b141 2024-09-13T09:18:38.723 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:38.723 INFO:teuthology.orchestra.run.smithi032.stdout:3935 read 0x27fea57 thru 0x2806aa2 (0x804c bytes) 2024-09-13T09:18:38.723 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:38.724 INFO:teuthology.orchestra.run.smithi032.stdout:3940 write 0x5d5bcf3 thru 0x5d68fdc (0xd2ea bytes) 2024-09-13T09:18:38.724 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:38.724 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:38.742 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:38.743 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:38.743 INFO:teuthology.orchestra.run.smithi032.stdout:3944 read 0x38ae0e0 thru 0x38b2784 (0x46a5 bytes) 2024-09-13T09:18:38.743 INFO:teuthology.orchestra.run.smithi032.stdout:3947 read 0xa144b95 thru 0xa14e4af (0x991b bytes) 2024-09-13T09:18:38.743 INFO:teuthology.orchestra.run.smithi032.stdout:3948 punch from 0x98e4114 to 0x98eb684, (0x7570 bytes) 2024-09-13T09:18:38.743 INFO:teuthology.orchestra.run.smithi032.stdout:3949 punch from 0x2a0c1c3 to 0x2a0e105, (0x1f42 bytes) 2024-09-13T09:18:38.743 INFO:teuthology.orchestra.run.smithi032.stdout:3950 punch from 0x109ae7a to 0x109eaf6, (0x3c7c bytes) 2024-09-13T09:18:38.744 INFO:teuthology.orchestra.run.smithi032.stdout:3951 read 0x472db44 thru 0x473acac (0xd169 bytes) 2024-09-13T09:18:38.744 INFO:teuthology.orchestra.run.smithi032.stdout:3952 read 0xabb4c4e thru 0xabc1313 (0xc6c6 bytes) 2024-09-13T09:18:38.744 INFO:teuthology.orchestra.run.smithi032.stdout:3953 write 0x35ea2a1 thru 0x35f0988 (0x66e8 bytes) 2024-09-13T09:18:38.763 INFO:teuthology.orchestra.run.smithi032.stdout:3954 read 0x298498c thru 0x298883e (0x3eb3 bytes) 2024-09-13T09:18:38.763 INFO:teuthology.orchestra.run.smithi032.stdout:3958 writesame 0x54a14aa thru 0x54a55f7 data_size 0xd(0x414e bytes) 2024-09-13T09:18:38.778 INFO:teuthology.orchestra.run.smithi032.stdout:3960 write 0x9922e60 thru 0x9932627 (0xf7c8 bytes) 2024-09-13T09:18:38.809 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:38.802+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:38.812 INFO:teuthology.orchestra.run.smithi032.stdout:3961 trunc from 0xc38b141 to 0x42cb952 2024-09-13T09:18:38.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:38.909 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:38.902+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:38.911 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:38.911 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:38.922 INFO:teuthology.orchestra.run.smithi032.stdout:3962 write 0x7d25fef thru 0x7d31574 (0xb586 bytes) 2024-09-13T09:18:38.924 INFO:teuthology.orchestra.run.smithi032.stdout:3963 read 0x15f2da thru 0x166ddc (0x7b03 bytes) 2024-09-13T09:18:38.926 INFO:teuthology.orchestra.run.smithi032.stdout:3964 read 0x1bb252 thru 0x1c1cc2 (0x6a71 bytes) 2024-09-13T09:18:38.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:38.927 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:38.927 INFO:teuthology.orchestra.run.smithi032.stdout:3966 read 0x150b00b thru 0x150d57c (0x2572 bytes) 2024-09-13T09:18:38.928 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:38.940 INFO:teuthology.orchestra.run.smithi032.stdout:3969 write 0x8a4f8e6 thru 0x8a53350 (0x3a6b bytes) 2024-09-13T09:18:38.959 INFO:teuthology.orchestra.run.smithi032.stdout:3970 write 0xb79c5a4 thru 0xb79dac7 (0x1524 bytes) 2024-09-13T09:18:38.962 INFO:teuthology.orchestra.run.smithi032.stdout:3971 punch from 0xb96512 to 0xb9cde3, (0x68d1 bytes) 2024-09-13T09:18:38.962 INFO:teuthology.orchestra.run.smithi032.stdout:3972 punch from 0x5ad63b0 to 0x5addfd9, (0x7c29 bytes) 2024-09-13T09:18:38.963 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:38.964 INFO:teuthology.orchestra.run.smithi032.stdout:3975 trunc from 0xb79dac8 to 0xcaf04e1 2024-09-13T09:18:38.994 INFO:teuthology.orchestra.run.smithi032.stdout:3977 trunc from 0xcaf04e1 to 0x1d41e13 2024-09-13T09:18:39.009 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:39.006+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:39.014 INFO:teuthology.orchestra.run.smithi032.stdout:3978 punch from 0x158ddd5 to 0x1593ed6, (0x6101 bytes) 2024-09-13T09:18:39.014 INFO:teuthology.orchestra.run.smithi032.stdout:3980 punch from 0x17e0857 to 0x17f0109, (0xf8b2 bytes) 2024-09-13T09:18:39.014 INFO:teuthology.orchestra.run.smithi032.stdout:3981 read 0x16f9bfd thru 0x16fb61d (0x1a21 bytes) 2024-09-13T09:18:39.015 INFO:teuthology.orchestra.run.smithi032.stdout:3982 punch from 0x1553289 to 0x1556996, (0x370d bytes) 2024-09-13T09:18:39.015 INFO:teuthology.orchestra.run.smithi032.stdout:3983 write 0xcfab32 thru 0xd01f46 (0x7415 bytes) 2024-09-13T09:18:39.047 INFO:teuthology.orchestra.run.smithi032.stdout:3985 write 0xd68fa1c thru 0xd696ee2 (0x74c7 bytes) 2024-09-13T09:18:39.049 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:39.050 INFO:teuthology.orchestra.run.smithi032.stdout:3988 read 0x6942180 thru 0x694fdbb (0xdc3c bytes) 2024-09-13T09:18:39.050 INFO:teuthology.orchestra.run.smithi032.stdout:3989 write 0x809525d thru 0x80a34b6 (0xe25a bytes) 2024-09-13T09:18:39.058 INFO:teuthology.orchestra.run.smithi032.stdout:3990 trunc from 0xd696ee3 to 0xa58dd87 2024-09-13T09:18:39.061 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:39.061 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:39.065 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:39.066 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:39.077 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:39.077 INFO:teuthology.orchestra.run.smithi032.stdout:3991 write 0x6ac8a95 thru 0x6ace64f (0x5bbb bytes) 2024-09-13T09:18:39.081 INFO:teuthology.orchestra.run.smithi032.stdout:3992 read 0x73201e4 thru 0x7325ad2 (0x58ef bytes) 2024-09-13T09:18:39.082 INFO:teuthology.orchestra.run.smithi032.stdout:3994 punch from 0x15f1114 to 0x15ff709, (0xe5f5 bytes) 2024-09-13T09:18:39.082 INFO:teuthology.orchestra.run.smithi032.stdout:3995 punch from 0x6cc4f19 to 0x6cca135, (0x521c bytes) 2024-09-13T09:18:39.082 INFO:teuthology.orchestra.run.smithi032.stdout:3997 read 0x889108a thru 0x889d005 (0xbf7c bytes) 2024-09-13T09:18:39.083 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:39.083 INFO:teuthology.orchestra.run.smithi032.stdout:3999 write 0x3e422c8 thru 0x3e4d398 (0xb0d1 bytes) 2024-09-13T09:18:39.083 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:39.084 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:39.086 INFO:teuthology.orchestra.run.smithi032.stdout:4000 punch from 0x332950e to 0x332daa7, (0x4599 bytes) 2024-09-13T09:18:39.087 INFO:teuthology.orchestra.run.smithi032.stdout:4002 writesame 0xd53d5d thru 0xd5d61d data_size 0x4f(0x98c1 bytes) 2024-09-13T09:18:39.090 INFO:teuthology.orchestra.run.smithi032.stdout:4003 punch from 0x1009759 to 0x10121bf, (0x8a66 bytes) 2024-09-13T09:18:39.091 INFO:teuthology.orchestra.run.smithi032.stdout:4004 punch from 0x548bc69 to 0x54925e5, (0x697c bytes) 2024-09-13T09:18:39.091 INFO:teuthology.orchestra.run.smithi032.stdout:4006 read 0x16ed4ed thru 0x16f18a1 (0x43b5 bytes) 2024-09-13T09:18:39.091 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:39.091 INFO:teuthology.orchestra.run.smithi032.stdout:4008 trunc from 0xa58dd87 to 0xaa8461e 2024-09-13T09:18:39.100 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:39.101 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:39.102 INFO:teuthology.orchestra.run.smithi032.stdout:4009 read 0x310dc7c thru 0x310ea1a (0xd9f bytes) 2024-09-13T09:18:39.103 INFO:teuthology.orchestra.run.smithi032.stdout:4011 punch from 0x8762e7c to 0x8769f37, (0x70bb bytes) 2024-09-13T09:18:39.103 INFO:teuthology.orchestra.run.smithi032.stdout:4013 write 0x3938fa2 thru 0x393e270 (0x52cf bytes) 2024-09-13T09:18:39.107 INFO:teuthology.orchestra.run.smithi032.stdout:4014 read 0x25b0d30 thru 0x25b4f3e (0x420f bytes) 2024-09-13T09:18:39.107 INFO:teuthology.orchestra.run.smithi032.stdout:4015 writesame 0x2abe49a thru 0x2ac78a7 data_size 0x3(0x940e bytes) 2024-09-13T09:18:39.110 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:39.106+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:39.119 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:39.119 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:39.122 INFO:teuthology.orchestra.run.smithi032.stdout:4016 clone 29 order 22 su 1048576 sc 10 2024-09-13T09:18:39.142 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:39.162 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:39.163 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:39.183 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:39.183 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:39.202 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:39.202 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:39.202 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:39.204 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:39.204 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:39.208 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:18:39.211 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:39.210+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:39.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:39.312 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:39.310+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:39.347 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone28 from 0xaa8461e (overlap 0x1d41e13) to 0xea0f09 2024-09-13T09:18:39.350 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:39.350 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:39.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:39.413 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:39.406+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:39.509 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-13T09:18:39.513 INFO:teuthology.orchestra.run.smithi032.stdout:4017 write 0x94c9254 thru 0x94cd931 (0x46de bytes) 2024-09-13T09:18:39.513 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:39.510+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:39.535 INFO:teuthology.orchestra.run.smithi032.stdout:4018 punch from 0x75ad8c8 to 0x75bca0f, (0xf147 bytes) 2024-09-13T09:18:39.540 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.003280633000003, 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:18:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:18:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:18:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:18:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:18:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:18:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:18:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:18:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:18:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:18:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:18:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:18:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32899 3.18 3:1fefe8db:::rbd_data.1185833655de.000000000000001e:16 [read 704512~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633733585000002, 2024-09-13T09:18:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046124828, 2024-09-13T09:18:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32899 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905908+0000", 2024-09-13T09:18:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905907+0000", 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905909+0000", 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905919+0000", 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0520000000000007e-06 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905923+0000", 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.951907+0000", 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.045983221999999997 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.951969+0000", 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2442999999999998e-05 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952032+0000", 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999998e-05 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633713648000001, 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:18:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:18:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:18:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:18:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:18:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:18:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:18:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:18:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633696326999999, 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:18:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:18:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633679386000001, 2024-09-13T09:18:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:18:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:18:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:39.564 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633662158, 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633625736999999, 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:18:39.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633572274999999, 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:18:39.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:18:39.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:18:39.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:18:39.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:18:39.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:39.577 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633550254999999, 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.581 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.63352562, 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:18:39.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.586 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.586 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:39.586 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633507096999999, 2024-09-13T09:18:39.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:18:39.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.586 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.586 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.586 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.588 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:18:39.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.588 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:18:39.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633489226000002, 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:18:39.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:18:39.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633471896, 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:39.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:18:39.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633454426, 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:18:39.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:18:39.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633437560000001, 2024-09-13T09:18:39.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:18:39.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:39.608 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633419773, 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:39.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:18:39.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.633375463, 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:18:39.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.400244975, 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.617 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:39.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.400228142, 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:18:39.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:18:39.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.625 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.400211784, 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:39.626 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:18:39.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:18:39.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:18:39.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:39.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:39.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:39.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:18:39.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:18:39.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.630 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:39.630 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:39.630 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:39.631 INFO:teuthology.orchestra.run.smithi032.stdout:4019 trunc from 0xaa8461e to 0x80f1a76 2024-09-13T09:18:39.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:39.633 INFO:teuthology.orchestra.run.smithi032.stdout:4020 writesame 0xc2956d9 thru 0xc29aafc data_size 0x1e(0x5424 bytes) 2024-09-13T09:18:39.633 INFO:teuthology.orchestra.run.smithi032.stdout:4021 write 0xc1ac703 thru 0xc1b5f39 (0x9837 bytes) 2024-09-13T09:18:39.633 INFO:teuthology.orchestra.run.smithi032.stdout:4022 read 0x5d428a6 thru 0x5d45dd5 (0x3530 bytes) 2024-09-13T09:18:39.633 INFO:teuthology.orchestra.run.smithi032.stdout:4024 writesame 0x79017e thru 0x79b6cf data_size 0x1(0xb552 bytes) 2024-09-13T09:18:39.634 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:39.626+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:39.694 INFO:teuthology.orchestra.run.smithi032.stdout:4026 writesame 0x17a3cde thru 0x17afa70 data_size 0x7(0xbd93 bytes) 2024-09-13T09:18:39.711 INFO:teuthology.orchestra.run.smithi032.stdout:4028 writesame 0x37aa99f thru 0x37af40e data_size 0x18(0x4a70 bytes) 2024-09-13T09:18:39.732 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:39.726+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:39.733 INFO:teuthology.orchestra.run.smithi032.stdout:4029 write 0xcbcdc34 thru 0xcbd00af (0x247c bytes) 2024-09-13T09:18:39.756 INFO:teuthology.orchestra.run.smithi032.stdout:4030 writesame 0xe5cd73f thru 0xe5d9fa7 data_size 0x1f(0xc869 bytes) 2024-09-13T09:18:39.757 INFO:teuthology.orchestra.run.smithi032.stdout:4031 read 0x96c3f1c thru 0x96d10bc (0xd1a1 bytes) 2024-09-13T09:18:39.758 INFO:teuthology.orchestra.run.smithi032.stdout:4032 read 0x5371200 thru 0x537dac1 (0xc8c2 bytes) 2024-09-13T09:18:39.758 INFO:teuthology.orchestra.run.smithi032.stdout:4033 writesame 0x1a71989 thru 0x1a71a23 data_size 0x5(0x9b bytes) 2024-09-13T09:18:39.771 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:39.771 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:39.772 INFO:teuthology.orchestra.run.smithi032.stdout:4036 read 0xb82d19d thru 0xb82f25b (0x20bf bytes) 2024-09-13T09:18:39.772 INFO:teuthology.orchestra.run.smithi032.stdout:4039 write 0x4dda91f thru 0x4de4429 (0x9b0b bytes) 2024-09-13T09:18:39.779 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:39.779 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:39.779 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:39.780 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:39.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:39.794 INFO:teuthology.orchestra.run.smithi032.stdout:4041 trunc from 0xe5d9fa8 to 0xa73d5c1 2024-09-13T09:18:39.815 INFO:teuthology.orchestra.run.smithi032.stdout:4042 read 0x270e81b thru 0x271acf8 (0xc4de bytes) 2024-09-13T09:18:39.817 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:39.817 INFO:teuthology.orchestra.run.smithi032.stdout:4045 writesame 0x21397bd thru 0x213a33f data_size 0x1(0xb83 bytes) 2024-09-13T09:18:39.820 INFO:teuthology.orchestra.run.smithi032.stdout:4046 read 0x89143a1 thru 0x891f90e (0xb56e bytes) 2024-09-13T09:18:39.837 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:39.834+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:39.841 INFO:teuthology.orchestra.run.smithi032.stdout:4047 write 0xd7f23e9 thru 0xd7fd576 (0xb18e bytes) 2024-09-13T09:18:39.857 INFO:teuthology.orchestra.run.smithi032.stdout:4048 trunc from 0xd7fd577 to 0xa80270f 2024-09-13T09:18:39.879 INFO:teuthology.orchestra.run.smithi032.stdout:4049 trunc from 0xa80270f to 0x9c047ef 2024-09-13T09:18:39.904 INFO:teuthology.orchestra.run.smithi032.stdout:4050 writesame 0x4048cc3 thru 0x40495b9 data_size 0x1(0x8f7 bytes) 2024-09-13T09:18:39.913 INFO:teuthology.orchestra.run.smithi032.stdout:4051 writesame 0x3e9f900 thru 0x3e9fae4 data_size 0x1(0x1e5 bytes) 2024-09-13T09:18:39.927 INFO:teuthology.orchestra.run.smithi032.stdout:4054 writesame 0x90dff5c thru 0x90ecdb6 data_size 0x1(0xce5b bytes) 2024-09-13T09:18:39.934 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:39.930+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:39.959 INFO:teuthology.orchestra.run.smithi032.stdout:4056 trunc from 0x9c047ef to 0x9e397b9 2024-09-13T09:18:39.967 INFO:teuthology.orchestra.run.smithi032.stdout:4058 read 0x58759e5 thru 0x587e43f (0x8a5b bytes) 2024-09-13T09:18:39.968 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:39.968 INFO:teuthology.orchestra.run.smithi032.stdout:4061 read 0x742c715 thru 0x7431049 (0x4935 bytes) 2024-09-13T09:18:39.969 INFO:teuthology.orchestra.run.smithi032.stdout:4062 punch from 0x3584f8d to 0x358e21e, (0x9291 bytes) 2024-09-13T09:18:39.970 INFO:teuthology.orchestra.run.smithi032.stdout:4063 write 0x89f9eaa thru 0x89fab91 (0xce8 bytes) 2024-09-13T09:18:39.973 INFO:teuthology.orchestra.run.smithi032.stdout:4064 punch from 0x29645ec to 0x2966525, (0x1f39 bytes) 2024-09-13T09:18:39.974 INFO:teuthology.orchestra.run.smithi032.stdout:4065 read 0x16292fe thru 0x1631970 (0x8673 bytes) 2024-09-13T09:18:39.974 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:39.975 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:39.975 INFO:teuthology.orchestra.run.smithi032.stdout:4069 punch from 0x53776f5 to 0x537f8c9, (0x81d4 bytes) 2024-09-13T09:18:39.975 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:39.976 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:39.976 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:39.977 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:39.977 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:39.977 INFO:teuthology.orchestra.run.smithi032.stdout:4073 trunc from 0x9e397b9 to 0xa435355 2024-09-13T09:18:39.986 INFO:teuthology.orchestra.run.smithi032.stdout:4074 clone 30 order 25 su 2097152 sc 11 2024-09-13T09:18:39.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:40.035 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:40.034+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:40.116 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:40.117 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:40.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:40.117 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:40.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.117 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4388 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.117 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:40.117 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.127704426999998, 2024-09-13T09:18:40.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069349509000000004, 2024-09-13T09:18:40.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4388 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985769+0000", 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985775+0000", 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5230000000000001e-06 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985794+0000", 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8425999999999999e-05 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985809+0000", 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.486e-05 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.054965+0000", 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069156463000000001 2024-09-13T09:18:40.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055036+0000", 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0780999999999999e-05 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055120+0000", 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3486000000000007e-05 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4389 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.127653903999999, 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069473833999999998, 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4389 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985820+0000", 2024-09-13T09:18:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985859+0000", 2024-09-13T09:18:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9279999999999997e-05 2024-09-13T09:18:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985881+0000", 2024-09-13T09:18:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2116999999999998e-05 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985896+0000", 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4698999999999999e-05 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055152+0000", 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069256568000000004 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055220+0000", 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8255000000000003e-05 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055294+0000", 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3858999999999999e-05 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4390 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.127558292000003, 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069563635999999998, 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4390 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985915+0000", 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985922+0000", 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985941+0000", 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8260000000000001e-05 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985955+0000", 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4409999999999999e-05 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055325+0000", 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069370290000000001 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055395+0000", 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0158000000000002e-05 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055480+0000", 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4340999999999998e-05 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.127507225000002, 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:18:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:18:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.127458488999999, 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:18:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:18:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:18:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.127390820000002, 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:18:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.127250611000001, 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:18:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.127159745, 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:18:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:18:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:18:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:18:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:18:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:18:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:18:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:18:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.127084885000002, 2024-09-13T09:18:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:18:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:18:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:18:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.127013709000003, 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:18:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:18:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.126964655999998, 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:18:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.126902919999999, 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:18:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:18:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.126853250000003, 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:18:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:18:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:18:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:18:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:18:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.126804731, 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:18:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.126756335000003, 2024-09-13T09:18:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:18:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:18:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.126709048000002, 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:18:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:18:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:18:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:18:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:18:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:18:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.126625646000001, 2024-09-13T09:18:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:18:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:18:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:18:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.126551919999997, 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:18:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.126483796999999, 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:18:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:18:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:18:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:18:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:18:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.126433329999998, 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:18:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:18:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:18:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:18:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:18:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:18:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:18:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:18:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.211 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:40.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:40.213 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:40.210+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:40.313 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:40.310+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:40.350 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:40.350 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:40.366 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:40.387 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone29 from 0xa435355 (overlap 0x80f1a76) to 0x26aee56 2024-09-13T09:18:40.417 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:40.414+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:40.507 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:40.507 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:40.515 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:40.514+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:40.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:40.617 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:40.614+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:40.652 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:40.652 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:40.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:40.652 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:40.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.652 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:18:40.653 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:40.653 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.963428238999995, 2024-09-13T09:18:40.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.21372158699999999, 2024-09-13T09:18:40.653 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.653 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:40.653 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.653 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.653 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.653 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10 2024-09-13T09:18:40.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.653 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685363+0000", 2024-09-13T09:18:40.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:40.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685372+0000", 2024-09-13T09:18:40.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4590000000000004e-06 2024-09-13T09:18:40.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685394+0000", 2024-09-13T09:18:40.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1616000000000001e-05 2024-09-13T09:18:40.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685407+0000", 2024-09-13T09:18:40.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3675e-05 2024-09-13T09:18:40.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685484+0000", 2024-09-13T09:18:40.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7065999999999997e-05 2024-09-13T09:18:40.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:18:40.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685493+0000", 2024-09-13T09:18:40.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8070000000000006e-06 2024-09-13T09:18:40.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690770+0000", 2024-09-13T09:18:40.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052763869999999996 2024-09-13T09:18:40.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690940+0000", 2024-09-13T09:18:40.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017001900000000001 2024-09-13T09:18:40.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.691238+0000", 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029794499999999998 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.692057+0000", 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081943000000000001 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:40.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.898933+0000", 2024-09-13T09:18:40.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.20687588200000001 2024-09-13T09:18:40.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:40.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899026+0000", 2024-09-13T09:18:40.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2777999999999998e-05 2024-09-13T09:18:40.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899086+0000", 2024-09-13T09:18:40.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0188e-05 2024-09-13T09:18:40.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:14590 3.1d 3:be60f898:::rbd_data.11853ecd5f5f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13672152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.107579799, 2024-09-13T09:18:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052557382, 2024-09-13T09:18:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.661 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 14590 2024-09-13T09:18:40.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.661 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:40.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541207+0000", 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562477+0000", 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021270155999999998 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562539+0000", 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1851000000000001e-05 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562565+0000", 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6194e-05 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562702+0000", 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013731299999999999 2024-09-13T09:18:40.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.563071+0000", 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036892899999999999 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.565167+0000", 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020961959999999998 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:40.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.585261+0000", 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020093443999999999 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593596+0000", 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083352849999999996 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593696+0000", 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100021 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593770+0000", 2024-09-13T09:18:40.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3889999999999999e-05 2024-09-13T09:18:40.666 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.666 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.666 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.666 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:14584 3.2 e39/22 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head v 39'64, mlcod=38'62)", 2024-09-13T09:18:40.666 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:40.666 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.105594152999998, 2024-09-13T09:18:40.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051546091000000002, 2024-09-13T09:18:40.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543197+0000", 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588691+0000", 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045493475999999998 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588711+0000", 2024-09-13T09:18:40.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0485000000000001e-05 2024-09-13T09:18:40.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588721+0000", 2024-09-13T09:18:40.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.499e-06 2024-09-13T09:18:40.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588766+0000", 2024-09-13T09:18:40.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5228000000000001e-05 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588788+0000", 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2503000000000001e-05 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594671+0000", 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058822809999999996 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594744+0000", 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3776000000000007e-05 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:22580 3.1c 3:39876c9f:::rbd_data.1185d6f8ec87.000000000000000f:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.796341363000003, 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050372896, 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:40.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.672 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.672 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.672 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22580 2024-09-13T09:18:40.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.672 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:40.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852449+0000", 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852462+0000", 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3756e-05 2024-09-13T09:18:40.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852495+0000", 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2435999999999999e-05 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852516+0000", 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1347000000000001e-05 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852644+0000", 2024-09-13T09:18:40.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012777200000000001 2024-09-13T09:18:40.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852784+0000", 2024-09-13T09:18:40.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013980500000000001 2024-09-13T09:18:40.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:40.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.860936+0000", 2024-09-13T09:18:40.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081526859999999993 2024-09-13T09:18:40.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:40.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902522+0000", 2024-09-13T09:18:40.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041585203000000001 2024-09-13T09:18:40.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:40.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902560+0000", 2024-09-13T09:18:40.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8603000000000003e-05 2024-09-13T09:18:40.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902708+0000", 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014740900000000001 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902824+0000", 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116519 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54746 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8011776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.417410887, 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041426971999999999, 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54746 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231381+0000", 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231384+0000", 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.599e-06 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231390+0000", 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231395+0000", 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0170000000000004e-06 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272716+0000", 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041321670999999997 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272767+0000", 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0447e-05 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272809+0000", 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1659999999999998e-05 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54986 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6348800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4065022880000004, 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041375935000000003, 2024-09-13T09:18:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54986 2024-09-13T09:18:40.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.683 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:40.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:40.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:40.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242292+0000", 2024-09-13T09:18:40.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3039999999999999e-06 2024-09-13T09:18:40.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242298+0000", 2024-09-13T09:18:40.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:18:40.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242303+0000", 2024-09-13T09:18:40.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8359999999999999e-06 2024-09-13T09:18:40.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283539+0000", 2024-09-13T09:18:40.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041235330000000001 2024-09-13T09:18:40.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283593+0000", 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4926000000000002e-05 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283666+0000", 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2718000000000004e-05 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54987 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6447104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4064851059999999, 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041604369000000002, 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54987 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:40.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:40.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242309+0000", 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1160000000000002e-06 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242317+0000", 2024-09-13T09:18:40.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9799999999999998e-06 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242323+0000", 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6439999999999997e-06 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283711+0000", 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388085999999998 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283766+0000", 2024-09-13T09:18:40.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5112999999999999e-05 2024-09-13T09:18:40.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283912+0000", 2024-09-13T09:18:40.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014575300000000001 2024-09-13T09:18:40.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.692 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.693 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54988 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6545408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.693 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:40.693 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4064652859999995, 2024-09-13T09:18:40.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041865404000000002, 2024-09-13T09:18:40.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54988 2024-09-13T09:18:40.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.693 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242329+0000", 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242335+0000", 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242341+0000", 2024-09-13T09:18:40.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9440000000000002e-06 2024-09-13T09:18:40.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283952+0000", 2024-09-13T09:18:40.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041610981999999998 2024-09-13T09:18:40.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284076+0000", 2024-09-13T09:18:40.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124356 2024-09-13T09:18:40.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284193+0000", 2024-09-13T09:18:40.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116345 2024-09-13T09:18:40.697 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.697 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.697 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.697 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54989 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6643712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.697 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:40.697 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.406447172, 2024-09-13T09:18:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042123657000000002, 2024-09-13T09:18:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54989 2024-09-13T09:18:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242348+0000", 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.655e-06 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242354+0000", 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2269999999999998e-06 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242360+0000", 2024-09-13T09:18:40.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6170000000000003e-06 2024-09-13T09:18:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284248+0000", 2024-09-13T09:18:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041887830000000001 2024-09-13T09:18:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284340+0000", 2024-09-13T09:18:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1358000000000002e-05 2024-09-13T09:18:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284469+0000", 2024-09-13T09:18:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012932300000000001 2024-09-13T09:18:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54990 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6742016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4064138819999998, 2024-09-13T09:18:40.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042278333000000001, 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54990 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242378+0000", 2024-09-13T09:18:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242381+0000", 2024-09-13T09:18:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0680000000000001e-06 2024-09-13T09:18:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242392+0000", 2024-09-13T09:18:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0762e-05 2024-09-13T09:18:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242398+0000", 2024-09-13T09:18:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1040000000000003e-06 2024-09-13T09:18:40.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284506+0000", 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042107983000000002 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284564+0000", 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.821e-05 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284657+0000", 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2588000000000002e-05 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54991 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6840320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4063890910000003, 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042489459, 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.707 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54991 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242406+0000", 2024-09-13T09:18:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:18:40.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242412+0000", 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5919999999999997e-06 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242425+0000", 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2713e-05 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284694+0000", 2024-09-13T09:18:40.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042269558999999998 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284784+0000", 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9999000000000003e-05 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284893+0000", 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108422 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54992 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6938624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4063627199999997, 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042636897999999999, 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54992 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242429+0000", 2024-09-13T09:18:40.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242432+0000", 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7599999999999998e-06 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242439+0000", 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5760000000000002e-06 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242448+0000", 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284935+0000", 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042486949000000003 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284991+0000", 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6471000000000002e-05 2024-09-13T09:18:40.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285067+0000", 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5543000000000005e-05 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54993 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7036928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4063398429999996, 2024-09-13T09:18:40.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042867959999999997, 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54993 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242452+0000", 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242455+0000", 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4360000000000001e-06 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242463+0000", 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8220000000000007e-06 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242468+0000", 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9479999999999999e-06 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285107+0000", 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042639415999999999 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285196+0000", 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8721000000000007e-05 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285321+0000", 2024-09-13T09:18:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012495 2024-09-13T09:18:40.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.721 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.721 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54994 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7135232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.721 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:40.721 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.406312679, 2024-09-13T09:18:40.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043009622999999997, 2024-09-13T09:18:40.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54994 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:40.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242482+0000", 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242486+0000", 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1049999999999997e-06 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242489+0000", 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.004e-06 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285354+0000", 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.04286479 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285413+0000", 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9778000000000003e-05 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285489+0000", 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6141000000000002e-05 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54995 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7233536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4063013079999998, 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043166641999999998, 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54995 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242492+0000", 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.437e-06 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242496+0000", 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242498+0000", 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5950000000000001e-06 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285528+0000", 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043029755000000003 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285585+0000", 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285658+0000", 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2713999999999995e-05 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54996 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7331840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.406291993, 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043333782000000001, 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54996 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242500+0000", 2024-09-13T09:18:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242502+0000", 2024-09-13T09:18:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3599999999999999e-06 2024-09-13T09:18:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242505+0000", 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1439999999999999e-06 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242507+0000", 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.605e-06 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285698+0000", 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043190911999999998 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285754+0000", 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5541999999999998e-05 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285834+0000", 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0426000000000006e-05 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54997 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4062829040000002, 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043496174999999998, 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54997 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242509+0000", 2024-09-13T09:18:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242511+0000", 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.313e-06 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242514+0000", 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242516+0000", 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3999999999999999e-06 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285875+0000", 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043358578000000002 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285929+0000", 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4602999999999999e-05 2024-09-13T09:18:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286006+0000", 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6429999999999998e-05 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54998 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7528448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4061902919999998, 2024-09-13T09:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043666469999999999, 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54998 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242518+0000", 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999161 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242606+0000", 2024-09-13T09:18:40.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7607000000000003e-05 2024-09-13T09:18:40.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242617+0000", 2024-09-13T09:18:40.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0882e-05 2024-09-13T09:18:40.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242624+0000", 2024-09-13T09:18:40.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9009999999999997e-06 2024-09-13T09:18:40.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286046+0000", 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043422308 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286106+0000", 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0285000000000003e-05 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286269+0000", 2024-09-13T09:18:40.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000162578 2024-09-13T09:18:40.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.744 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.744 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:55005 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8216576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.744 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.245973+0000", 2024-09-13T09:18:40.744 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4028199350000001, 2024-09-13T09:18:40.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041347676, 2024-09-13T09:18:40.744 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.744 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.744 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 55005 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245973+0000", 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245973+0000", 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245972+0000", 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245974+0000", 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2110000000000001e-06 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245982+0000", 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2219999999999996e-06 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245987+0000", 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9139999999999999e-06 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287209+0000", 2024-09-13T09:18:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222560999999998 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287265+0000", 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6081000000000001e-05 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287320+0000", 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4972999999999997e-05 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.748 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:55006 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8314880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4028016240000003, 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041461771000000001, 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 55006 2024-09-13T09:18:40.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245993+0000", 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2869999999999999e-06 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245999+0000", 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0159999999999997e-06 2024-09-13T09:18:40.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.246004+0000", 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287345+0000", 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041341217 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:40.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287398+0000", 2024-09-13T09:18:40.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2840000000000002e-05 2024-09-13T09:18:40.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:40.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:40.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:40.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287453+0000", 2024-09-13T09:18:40.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4746000000000001e-05 2024-09-13T09:18:40.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.753 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:40.753 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:40.753 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:40.755 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-13T09:18:40.755 INFO:teuthology.orchestra.run.smithi032.stdout:4076 trunc from 0xa435355 to 0x3bcb15e 2024-09-13T09:18:40.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:40.756 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:40.754+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:40.856 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:40.854+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:40.865 INFO:teuthology.orchestra.run.smithi032.stdout:4077 punch from 0x1f2030d to 0x1f2afce, (0xacc1 bytes) 2024-09-13T09:18:40.865 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:40.889 INFO:teuthology.orchestra.run.smithi032.stdout:4080 writesame 0xa17f004 thru 0xa180179 data_size 0xf(0x1176 bytes) 2024-09-13T09:18:40.892 INFO:teuthology.orchestra.run.smithi032.stdout:4081 writesame 0x76067cd thru 0x7616411 data_size 0x67(0xfc45 bytes) 2024-09-13T09:18:40.896 INFO:teuthology.orchestra.run.smithi032.stdout:4082 read 0x63a7f81 thru 0x63acd73 (0x4df3 bytes) 2024-09-13T09:18:40.897 INFO:teuthology.orchestra.run.smithi032.stdout:4084 writesame 0x78ff7b0 thru 0x7906816 data_size 0x1(0x7067 bytes) 2024-09-13T09:18:40.916 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:40.916 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:40.916 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:40.916 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:40.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:40.933 INFO:teuthology.orchestra.run.smithi032.stdout:4085 read 0x1583086 thru 0x1587106 (0x4081 bytes) 2024-09-13T09:18:40.934 INFO:teuthology.orchestra.run.smithi032.stdout:4086 punch from 0x6ae0876 to 0x6ae545f, (0x4be9 bytes) 2024-09-13T09:18:40.935 INFO:teuthology.orchestra.run.smithi032.stdout:4087 trunc from 0xa18017a to 0xa649449 2024-09-13T09:18:40.965 INFO:teuthology.orchestra.run.smithi032.stdout:4088 trunc from 0xa649449 to 0x36a4111 2024-09-13T09:18:40.978 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:40.974+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:40.980 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:40.980 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xec5e3ed 2024-09-13T09:18:41.008 INFO:teuthology.orchestra.run.smithi032.stdout:4090 trunc from 0x36a4111 to 0xec5e3ed 2024-09-13T09:18:41.021 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.021 INFO:teuthology.orchestra.run.smithi032.stdout:4092 read 0x609461c thru 0x60a3aa5 (0xf48a bytes) 2024-09-13T09:18:41.022 INFO:teuthology.orchestra.run.smithi032.stdout:4093 write 0x8aa85ef thru 0x8ab06b5 (0x80c7 bytes) 2024-09-13T09:18:41.026 INFO:teuthology.orchestra.run.smithi032.stdout:4095 read 0x1b0cac3 thru 0x1b11b75 (0x50b3 bytes) 2024-09-13T09:18:41.027 INFO:teuthology.orchestra.run.smithi032.stdout:4096 punch from 0xd0df0ab to 0xd0e5086, (0x5fdb bytes) 2024-09-13T09:18:41.058 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:41.054+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:41.058 INFO:teuthology.orchestra.run.smithi032.stdout:4097 trunc from 0xec5e3ed to 0x20c68f1 2024-09-13T09:18:41.065 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:41.065 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:41.065 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:41.065 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:41.066 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:41.080 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.081 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:41.102 INFO:teuthology.orchestra.run.smithi032.stdout:4099 writesame 0x60bd630 thru 0x60c685a data_size 0x1(0x922b bytes) 2024-09-13T09:18:41.104 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.104 INFO:teuthology.orchestra.run.smithi032.stdout:4101 write 0x6a5368 thru 0x6a6fa7 (0x1c40 bytes) 2024-09-13T09:18:41.124 INFO:teuthology.orchestra.run.smithi032.stdout:4102 writesame 0x7fb7395 thru 0x7fc291f data_size 0xd(0xb58b bytes) 2024-09-13T09:18:41.143 INFO:teuthology.orchestra.run.smithi032.stdout:4104 trunc from 0x7fc2920 to 0xdba4b60 2024-09-13T09:18:41.153 INFO:teuthology.orchestra.run.smithi032.stdout:4106 write 0x4ea4cf4 thru 0x4eb4bc4 (0xfed1 bytes) 2024-09-13T09:18:41.156 INFO:teuthology.orchestra.run.smithi032.stdout:4107 writesame 0x8a3352d thru 0x8a43055 data_size 0x1(0xfb29 bytes) 2024-09-13T09:18:41.159 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:41.158+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:41.202 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.202 INFO:teuthology.orchestra.run.smithi032.stdout:4109 read 0xa3327e4 thru 0xa335d74 (0x3591 bytes) 2024-09-13T09:18:41.203 INFO:teuthology.orchestra.run.smithi032.stdout:4110 read 0x1275b18 thru 0x12792ea (0x37d3 bytes) 2024-09-13T09:18:41.224 INFO:teuthology.orchestra.run.smithi032.stdout:4111 trunc from 0xdba4b60 to 0x66e4d88 2024-09-13T09:18:41.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:18:41.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:18:41.240 INFO:teuthology.orchestra.run.smithi032.stdout:4112 read 0x6569691 thru 0x6573952 (0xa2c2 bytes) 2024-09-13T09:18:41.240 INFO:teuthology.orchestra.run.smithi032.stdout:4114 writesame 0x2e1d28a thru 0x2e20be1 data_size 0x14(0x3958 bytes) 2024-09-13T09:18:41.245 INFO:teuthology.orchestra.run.smithi032.stdout:4116 read 0x6236630 thru 0x6241010 (0xa9e1 bytes) 2024-09-13T09:18:41.245 INFO:teuthology.orchestra.run.smithi032.stdout:4118 read 0x5ffcdb6 thru 0x5fff952 (0x2b9d bytes) 2024-09-13T09:18:41.246 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.246 INFO:teuthology.orchestra.run.smithi032.stdout:4121 punch from 0x267a9a0 to 0x267e47e, (0x3ade bytes) 2024-09-13T09:18:41.255 INFO:teuthology.orchestra.run.smithi032.stdout:4122 trunc from 0x66e4d88 to 0x2d2d392 2024-09-13T09:18:41.259 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:41.258+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:41.265 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:41.265 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:41.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:41.265 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:41.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.265 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:50082 3.10 3:0ec43ecb:::rbd_object_map.1185172eb410:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T09:18:41.266 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:41.266 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.898822296000001, 2024-09-13T09:18:41.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80943783999999996, 2024-09-13T09:18:41.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:41.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 50082 2024-09-13T09:18:41.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.266 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362619+0000", 2024-09-13T09:18:41.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362632+0000", 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3338e-05 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362657+0000", 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5207000000000001e-05 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362669+0000", 2024-09-13T09:18:41.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1768e-05 2024-09-13T09:18:41.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362738+0000", 2024-09-13T09:18:41.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9486999999999994e-05 2024-09-13T09:18:41.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:18:41.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362748+0000", 2024-09-13T09:18:41.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0010000000000001e-05 2024-09-13T09:18:41.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169473+0000", 2024-09-13T09:18:41.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80672482300000004 2024-09-13T09:18:41.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:41.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169884+0000", 2024-09-13T09:18:41.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041114299999999999 2024-09-13T09:18:41.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:41.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.170206+0000", 2024-09-13T09:18:41.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032127700000000002 2024-09-13T09:18:41.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:41.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171827+0000", 2024-09-13T09:18:41.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016217880000000001 2024-09-13T09:18:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171871+0000", 2024-09-13T09:18:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:18:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171967+0000", 2024-09-13T09:18:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5752000000000001e-05 2024-09-13T09:18:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.172059+0000", 2024-09-13T09:18:41.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2082000000000004e-05 2024-09-13T09:18:41.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.273 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.273 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56772 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 4112384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.273 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:41.273 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.831134177, 2024-09-13T09:18:41.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054477114, 2024-09-13T09:18:41.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:41.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56772 2024-09-13T09:18:41.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.274 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:41.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:41.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430309+0000", 2024-09-13T09:18:41.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:41.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430312+0000", 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.255e-06 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430319+0000", 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4760000000000001e-06 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430324+0000", 2024-09-13T09:18:41.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9799999999999998e-06 2024-09-13T09:18:41.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484634+0000", 2024-09-13T09:18:41.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054310031000000002 2024-09-13T09:18:41.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:41.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484697+0000", 2024-09-13T09:18:41.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3355999999999994e-05 2024-09-13T09:18:41.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484787+0000", 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9350000000000003e-05 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56785 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5390336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8273506949999998, 2024-09-13T09:18:41.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054655007999999998, 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56785 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434094+0000", 2024-09-13T09:18:41.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.471e-06 2024-09-13T09:18:41.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434097+0000", 2024-09-13T09:18:41.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0740000000000001e-06 2024-09-13T09:18:41.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434100+0000", 2024-09-13T09:18:41.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7829999999999999e-06 2024-09-13T09:18:41.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488422+0000", 2024-09-13T09:18:41.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054321855000000002 2024-09-13T09:18:41.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:41.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488563+0000", 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014154 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488748+0000", 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018461399999999999 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56786 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8273413129999998, 2024-09-13T09:18:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054950539999999999, 2024-09-13T09:18:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56786 2024-09-13T09:18:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434104+0000", 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4020000000000001e-06 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434107+0000", 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.072e-06 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434109+0000", 2024-09-13T09:18:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.717e-06 2024-09-13T09:18:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488795+0000", 2024-09-13T09:18:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054685661000000003 2024-09-13T09:18:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488869+0000", 2024-09-13T09:18:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4265999999999997e-05 2024-09-13T09:18:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489053+0000", 2024-09-13T09:18:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018374299999999999 2024-09-13T09:18:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56787 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5586944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8273310540000001, 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055115725999999997, 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56787 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434112+0000", 2024-09-13T09:18:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434114+0000", 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.308e-06 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434117+0000", 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434119+0000", 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.588e-06 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489097+0000", 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054977824000000002 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489146+0000", 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8503e-05 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489228+0000", 2024-09-13T09:18:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2694000000000005e-05 2024-09-13T09:18:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56788 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8273221629999998, 2024-09-13T09:18:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055437562000000003, 2024-09-13T09:18:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56788 2024-09-13T09:18:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434121+0000", 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434123+0000", 2024-09-13T09:18:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2190000000000001e-06 2024-09-13T09:18:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434126+0000", 2024-09-13T09:18:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:18:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434128+0000", 2024-09-13T09:18:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8250000000000001e-06 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489293+0000", 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055164092999999997 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489428+0000", 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013572599999999999 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489559+0000", 2024-09-13T09:18:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013086 2024-09-13T09:18:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56789 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8273130139999996, 2024-09-13T09:18:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055674077000000002, 2024-09-13T09:18:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56789 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434132+0000", 2024-09-13T09:18:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.302e-06 2024-09-13T09:18:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434136+0000", 2024-09-13T09:18:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-13T09:18:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434139+0000", 2024-09-13T09:18:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:18:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489583+0000", 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055443994000000003 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489699+0000", 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116611 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489805+0000", 2024-09-13T09:18:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105331 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56790 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.827302639, 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055716839999999997, 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56790 2024-09-13T09:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434142+0000", 2024-09-13T09:18:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2559999999999999e-06 2024-09-13T09:18:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434145+0000", 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.96e-06 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434148+0000", 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6479999999999999e-06 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489851+0000", 2024-09-13T09:18:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055702792000000001 2024-09-13T09:18:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489858+0000", 2024-09-13T09:18:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3900000000000004e-06 2024-09-13T09:18:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56791 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8272938669999998, 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055740029000000003, 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56791 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434151+0000", 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434154+0000", 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0139999999999999e-06 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434157+0000", 2024-09-13T09:18:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6699999999999998e-06 2024-09-13T09:18:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489884+0000", 2024-09-13T09:18:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055727511 2024-09-13T09:18:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489890+0000", 2024-09-13T09:18:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7200000000000003e-06 2024-09-13T09:18:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56792 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.827284938, 2024-09-13T09:18:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055760363, 2024-09-13T09:18:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56792 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434163+0000", 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434167+0000", 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.292e-06 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434169+0000", 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489914+0000", 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055744415999999998 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489919+0000", 2024-09-13T09:18:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:18:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56793 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8272714180000005, 2024-09-13T09:18:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055774049999999999, 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56793 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434173+0000", 2024-09-13T09:18:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-06 2024-09-13T09:18:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434177+0000", 2024-09-13T09:18:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1070000000000001e-06 2024-09-13T09:18:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434179+0000", 2024-09-13T09:18:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.666e-06 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489941+0000", 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055762071000000003 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489946+0000", 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56794 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8272623100000001, 2024-09-13T09:18:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055791704999999997, 2024-09-13T09:18:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56794 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434182+0000", 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1629999999999999e-06 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434185+0000", 2024-09-13T09:18:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9950000000000001e-06 2024-09-13T09:18:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434188+0000", 2024-09-13T09:18:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:18:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489968+0000", 2024-09-13T09:18:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055779789000000003 2024-09-13T09:18:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489973+0000", 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2800000000000003e-06 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56795 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8272533739999997, 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055809612000000001, 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56795 2024-09-13T09:18:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434191+0000", 2024-09-13T09:18:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.212e-06 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434194+0000", 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.078e-06 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434198+0000", 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9820000000000002e-06 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489995+0000", 2024-09-13T09:18:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055796363000000002 2024-09-13T09:18:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490000+0000", 2024-09-13T09:18:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1839999999999998e-06 2024-09-13T09:18:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56796 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8272431129999998, 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055825859999999998, 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56796 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434200+0000", 2024-09-13T09:18:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434202+0000", 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.204e-06 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434205+0000", 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9689999999999999e-06 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434207+0000", 2024-09-13T09:18:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-13T09:18:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490021+0000", 2024-09-13T09:18:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055813929999999998 2024-09-13T09:18:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490027+0000", 2024-09-13T09:18:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.259e-06 2024-09-13T09:18:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56797 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8272343939999995, 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055843649000000002, 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56797 2024-09-13T09:18:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434210+0000", 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2389999999999999e-06 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434213+0000", 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434216+0000", 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7800000000000001e-06 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490048+0000", 2024-09-13T09:18:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055831981000000003 2024-09-13T09:18:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490053+0000", 2024-09-13T09:18:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8640000000000004e-06 2024-09-13T09:18:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56807 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8255542360000003, 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054407720999999999, 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56807 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435889+0000", 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435891+0000", 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.217e-06 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435894+0000", 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.067e-06 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435896+0000", 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490292+0000", 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054395840000000001 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490297+0000", 2024-09-13T09:18:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56808 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8255454100000001, 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05442524, 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56808 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435900+0000", 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435903+0000", 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435905+0000", 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490319+0000", 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054413396000000003 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490324+0000", 2024-09-13T09:18:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:18:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56809 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8255353540000003, 2024-09-13T09:18:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054442786, 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56809 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435910+0000", 2024-09-13T09:18:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:18:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435913+0000", 2024-09-13T09:18:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1470000000000001e-06 2024-09-13T09:18:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435915+0000", 2024-09-13T09:18:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6769999999999999e-06 2024-09-13T09:18:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490346+0000", 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054430574000000002 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490351+0000", 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2700000000000004e-06 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56810 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8255260550000001, 2024-09-13T09:18:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054459855000000001, 2024-09-13T09:18:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56810 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435917+0000", 2024-09-13T09:18:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435919+0000", 2024-09-13T09:18:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.457e-06 2024-09-13T09:18:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435924+0000", 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435929+0000", 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8300000000000003e-06 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490373+0000", 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054443841999999999 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490378+0000", 2024-09-13T09:18:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8509999999999998e-06 2024-09-13T09:18:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56811 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.8254969540000001, 2024-09-13T09:18:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054456683999999998, 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56811 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435946+0000", 2024-09-13T09:18:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435949+0000", 2024-09-13T09:18:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:18:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435956+0000", 2024-09-13T09:18:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8719999999999996e-06 2024-09-13T09:18:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435962+0000", 2024-09-13T09:18:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000004e-06 2024-09-13T09:18:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490398+0000", 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054436824000000002 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490403+0000", 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0710000000000002e-06 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:41.362 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:41.364 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:41.364 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:41.358+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:41.365 INFO:teuthology.orchestra.run.smithi032.stdout:4127 read 0x11c007c thru 0x11c4290 (0x4215 bytes) 2024-09-13T09:18:41.365 INFO:teuthology.orchestra.run.smithi032.stdout:4128 trunc from 0x2d2d392 to 0xcc4ff01 2024-09-13T09:18:41.365 INFO:teuthology.orchestra.run.smithi032.stdout:4129 punch from 0x79a7a0 to 0x7aa411, (0xfc71 bytes) 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:4131 write 0xe1a56b8 thru 0xe1b3d08 (0xe651 bytes) 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:4133 read 0x446998c thru 0x44709d3 (0x7048 bytes) 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:4135 read 0x59793a0 thru 0x597972d (0x38e bytes) 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:4136 punch from 0xa1893c0 to 0xa189768, (0x3a8 bytes) 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:4137 trunc from 0xe1b3d09 to 0x699b9aa 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:4140 writesame 0xd99283 thru 0xd9d375 data_size 0x1(0x40f3 bytes) 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:4141 punch from 0xbdec6a to 0xbe2f04, (0x429a bytes) 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:4143 punch from 0x1b33c7b to 0x1b430e1, (0xf466 bytes) 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:4145 writesame 0x6b97050 thru 0x6ba6690 data_size 0x1(0xf641 bytes) 2024-09-13T09:18:41.366 INFO:teuthology.orchestra.run.smithi032.stdout:4146 punch from 0x3d8b89e to 0x3d8e258, (0x29ba bytes) 2024-09-13T09:18:41.367 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.367 INFO:teuthology.orchestra.run.smithi032.stdout:4150 read 0x455952a thru 0x455e8b3 (0x538a bytes) 2024-09-13T09:18:41.367 INFO:teuthology.orchestra.run.smithi032.stdout:4152 writesame 0x4aac0ce thru 0x4ab75d0 data_size 0x1(0xb503 bytes) 2024-09-13T09:18:41.396 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:41.396 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:41.412 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:41.412 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:41.416 INFO:teuthology.orchestra.run.smithi032.stdout:4153 read 0x2fdbee3 thru 0x2fe7c3b (0xbd59 bytes) 2024-09-13T09:18:41.416 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.416 INFO:teuthology.orchestra.run.smithi032.stdout:4157 read 0xa586db thru 0xa63e9b (0xb7c1 bytes) 2024-09-13T09:18:41.421 INFO:teuthology.orchestra.run.smithi032.stdout:4158 trunc from 0x6ba6691 to 0x9c83924 2024-09-13T09:18:41.429 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:41.443 INFO:teuthology.orchestra.run.smithi032.stdout:4159 write 0xa898fc5 thru 0xa8a8a98 (0xfad4 bytes) 2024-09-13T09:18:41.446 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:41.456 INFO:teuthology.orchestra.run.smithi032.stdout:4160 writesame 0xb359991 thru 0xb35b040 data_size 0x30(0x16b0 bytes) 2024-09-13T09:18:41.457 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.464 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:41.458+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:41.466 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:41.466 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:41.472 INFO:teuthology.orchestra.run.smithi032.stdout:4164 trunc from 0xb35b041 to 0x184d8ba 2024-09-13T09:18:41.488 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:41.488 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:41.495 INFO:teuthology.orchestra.run.smithi032.stdout:4165 writesame 0x4302932 thru 0x4309f4a data_size 0x1(0x7619 bytes) 2024-09-13T09:18:41.497 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.508 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:41.514 INFO:teuthology.orchestra.run.smithi032.stdout:4168 writesame 0xab3e63e thru 0xab43f6e data_size 0x3b(0x5931 bytes) 2024-09-13T09:18:41.522 INFO:teuthology.orchestra.run.smithi032.stdout:4170 trunc from 0xab43f6f to 0x6b1aa65 2024-09-13T09:18:41.529 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:41.536 INFO:teuthology.orchestra.run.smithi032.stdout:4172 read 0x25eac15 thru 0x25f80c8 (0xd4b4 bytes) 2024-09-13T09:18:41.540 INFO:teuthology.orchestra.run.smithi032.stdout:4173 trunc from 0x6b1aa65 to 0x9103fae 2024-09-13T09:18:41.554 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:41.554 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:41.564 INFO:teuthology.orchestra.run.smithi032.stdout:4175 write 0xdb7f3ff thru 0xdb8b7a6 (0xc3a8 bytes) 2024-09-13T09:18:41.564 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:41.562+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:41.566 INFO:teuthology.orchestra.run.smithi032.stdout:4176 write 0xc9ba4b8 thru 0xc9bbba6 (0x16ef bytes) 2024-09-13T09:18:41.568 INFO:teuthology.orchestra.run.smithi032.stdout:4177 write 0x8fbf914 thru 0x8fcd30e (0xd9fb bytes) 2024-09-13T09:18:41.570 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:41.576 INFO:teuthology.orchestra.run.smithi032.stdout:4178 writesame 0xdc7d485 thru 0xdc83de5 data_size 0x1(0x6961 bytes) 2024-09-13T09:18:41.578 INFO:teuthology.orchestra.run.smithi032.stdout:4180 writesame 0xdb63f2 thru 0xdbd1b6 data_size 0x33(0x6dc5 bytes) 2024-09-13T09:18:41.581 INFO:teuthology.orchestra.run.smithi032.stdout:4181 read 0x9114dee thru 0x911be59 (0x706c bytes) 2024-09-13T09:18:41.581 INFO:teuthology.orchestra.run.smithi032.stdout:4183 read 0x53098a1 thru 0x53109cd (0x712d bytes) 2024-09-13T09:18:41.582 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.582 INFO:teuthology.orchestra.run.smithi032.stdout:4185 punch from 0xcd490eb to 0xcd4d73d, (0x4652 bytes) 2024-09-13T09:18:41.582 INFO:teuthology.orchestra.run.smithi032.stdout:4186 write 0xb8211aa thru 0xb8263e4 (0x523b bytes) 2024-09-13T09:18:41.584 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.584 INFO:teuthology.orchestra.run.smithi032.stdout:4188 write 0x4a5fb35 thru 0x4a6836c (0x8838 bytes) 2024-09-13T09:18:41.596 INFO:teuthology.orchestra.run.smithi032.stdout:4189 write 0xe31c5a7 thru 0xe325409 (0x8e63 bytes) 2024-09-13T09:18:41.598 INFO:teuthology.orchestra.run.smithi032.stdout:4190 write 0x3ae4752 thru 0x3aeadfd (0x66ac bytes) 2024-09-13T09:18:41.608 INFO:teuthology.orchestra.run.smithi032.stdout:4191 trunc from 0xe32540a to 0x9b9ddf8 2024-09-13T09:18:41.624 INFO:teuthology.orchestra.run.smithi032.stdout:4193 trunc from 0x9b9ddf8 to 0xd90fefe 2024-09-13T09:18:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:4194 writesame 0x7170bca thru 0x717dcef data_size 0x13(0xd126 bytes) 2024-09-13T09:18:41.638 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.638 INFO:teuthology.orchestra.run.smithi032.stdout:4196 read 0x458a1e thru 0x468045 (0xf628 bytes) 2024-09-13T09:18:41.639 INFO:teuthology.orchestra.run.smithi032.stdout:4197 punch from 0x7ce60d0 to 0x7cee090, (0x7fc0 bytes) 2024-09-13T09:18:41.639 INFO:teuthology.orchestra.run.smithi032.stdout:4200 read 0x8a2d39e thru 0x8a35fb6 (0x8c19 bytes) 2024-09-13T09:18:41.640 INFO:teuthology.orchestra.run.smithi032.stdout:4201 read 0x599edfc thru 0x59ae78c (0xf991 bytes) 2024-09-13T09:18:41.640 INFO:teuthology.orchestra.run.smithi032.stdout:4202 write 0x95a298b thru 0x95b0e16 (0xe48c bytes) 2024-09-13T09:18:41.653 INFO:teuthology.orchestra.run.smithi032.stdout:4204 trunc from 0xd90fefe to 0x95418d7 2024-09-13T09:18:41.666 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:41.662+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:4205 trunc from 0x95418d7 to 0x2d1a3d9 2024-09-13T09:18:41.721 INFO:teuthology.orchestra.run.smithi032.stdout:4206 write 0xda880d7 thru 0xda8f81a (0x7744 bytes) 2024-09-13T09:18:41.724 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.724 INFO:teuthology.orchestra.run.smithi032.stdout:4208 read 0x485e754 thru 0x4864822 (0x60cf bytes) 2024-09-13T09:18:41.724 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.725 INFO:teuthology.orchestra.run.smithi032.stdout:4210 punch from 0x25d5308 to 0x25e1ee4, (0xcbdc bytes) 2024-09-13T09:18:41.725 INFO:teuthology.orchestra.run.smithi032.stdout:4211 punch from 0x70eaad3 to 0x70f3dd8, (0x9305 bytes) 2024-09-13T09:18:41.725 INFO:teuthology.orchestra.run.smithi032.stdout:4213 read 0xd28cf85 thru 0xd2909b3 (0x3a2f bytes) 2024-09-13T09:18:41.725 INFO:teuthology.orchestra.run.smithi032.stdout:4214 writesame 0x5add30f thru 0x5addb61 data_size 0x1(0x853 bytes) 2024-09-13T09:18:41.728 INFO:teuthology.orchestra.run.smithi032.stdout:4215 writesame 0xa86079e thru 0xa86b06c data_size 0x43(0xa8cf bytes) 2024-09-13T09:18:41.731 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.731 INFO:teuthology.orchestra.run.smithi032.stdout:4219 write 0x6358c5a thru 0x6368bfb (0xffa2 bytes) 2024-09-13T09:18:41.733 INFO:teuthology.orchestra.run.smithi032.stdout:4220 write 0x262b2f2 thru 0x2637918 (0xc627 bytes) 2024-09-13T09:18:41.735 INFO:teuthology.orchestra.run.smithi032.stdout:4223 write 0x9ff8c6a thru 0x9fffb95 (0x6f2c bytes) 2024-09-13T09:18:41.738 INFO:teuthology.orchestra.run.smithi032.stdout:4225 read 0x8e18b34 thru 0x8e1f2a3 (0x6770 bytes) 2024-09-13T09:18:41.739 INFO:teuthology.orchestra.run.smithi032.stdout:4228 read 0x1e51dc thru 0x1f29d4 (0xd7f9 bytes) 2024-09-13T09:18:41.739 INFO:teuthology.orchestra.run.smithi032.stdout:4229 punch from 0x22952c1 to 0x22a223b, (0xcf7a bytes) 2024-09-13T09:18:41.739 INFO:teuthology.orchestra.run.smithi032.stdout:4230 read 0x20d46f3 thru 0x20d9fa3 (0x58b1 bytes) 2024-09-13T09:18:41.740 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:41.740 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:41.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:41.756 INFO:teuthology.orchestra.run.smithi032.stdout:4231 trunc from 0xda8f81b to 0x319a63d 2024-09-13T09:18:41.767 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:41.766+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:41.789 INFO:teuthology.orchestra.run.smithi032.stdout:4233 trunc from 0x319a63d to 0xd1984b8 2024-09-13T09:18:41.800 INFO:teuthology.orchestra.run.smithi032.stdout:4234 read 0x5e4a0ad thru 0x5e56c66 (0xcbba bytes) 2024-09-13T09:18:41.801 INFO:teuthology.orchestra.run.smithi032.stdout:4235 write 0x671e57d thru 0x6727220 (0x8ca4 bytes) 2024-09-13T09:18:41.804 INFO:teuthology.orchestra.run.smithi032.stdout:4236 writesame 0x3b6ecf2 thru 0x3b72b49 data_size 0x3(0x3e58 bytes) 2024-09-13T09:18:41.809 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.831 INFO:teuthology.orchestra.run.smithi032.stdout:4238 trunc from 0xd1984b8 to 0x3228f5f 2024-09-13T09:18:41.845 INFO:teuthology.orchestra.run.smithi032.stdout:4240 read 0x101e239 thru 0x102109f (0x2e67 bytes) 2024-09-13T09:18:41.846 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.867 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:41.866+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:41.871 INFO:teuthology.orchestra.run.smithi032.stdout:4243 trunc from 0x3228f5f to 0xe7db733 2024-09-13T09:18:41.886 INFO:teuthology.orchestra.run.smithi032.stdout:4244 writesame 0x9827ad5 thru 0x982becf data_size 0x1(0x43fb bytes) 2024-09-13T09:18:41.905 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.906 INFO:teuthology.orchestra.run.smithi032.stdout:4246 read 0xa5d1f91 thru 0xa5dc5dc (0xa64c bytes) 2024-09-13T09:18:41.907 INFO:teuthology.orchestra.run.smithi032.stdout:4247 punch from 0x73f3ead to 0x7400f26, (0xd079 bytes) 2024-09-13T09:18:41.907 INFO:teuthology.orchestra.run.smithi032.stdout:4248 punch from 0x699399f to 0x699bb88, (0x81e9 bytes) 2024-09-13T09:18:41.907 INFO:teuthology.orchestra.run.smithi032.stdout:4249 writesame 0xc093fd0 thru 0xc09a5fb data_size 0x1a(0x662c bytes) 2024-09-13T09:18:41.911 INFO:teuthology.orchestra.run.smithi032.stdout:4250 punch from 0x728ffd9 to 0x7290a9c, (0xac3 bytes) 2024-09-13T09:18:41.911 INFO:teuthology.orchestra.run.smithi032.stdout:4251 write 0x8e8a7c5 thru 0x8e95fb7 (0xb7f3 bytes) 2024-09-13T09:18:41.914 INFO:teuthology.orchestra.run.smithi032.stdout:4252 writesame 0x3c4e237 thru 0x3c52c65 data_size 0x1(0x4a2f bytes) 2024-09-13T09:18:41.934 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:41.935 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:41.940 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:41.962 INFO:teuthology.orchestra.run.smithi032.stdout:4256 trunc from 0xe7db733 to 0x664b878 2024-09-13T09:18:41.968 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:41.962+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:41.974 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:41.974 INFO:teuthology.orchestra.run.smithi032.stdout:4262 read 0x4cc2d81 thru 0x4cca145 (0x73c5 bytes) 2024-09-13T09:18:41.984 INFO:teuthology.orchestra.run.smithi032.stdout:4263 write 0x6c3626e thru 0x6c383c5 (0x2158 bytes) 2024-09-13T09:18:41.987 INFO:teuthology.orchestra.run.smithi032.stdout:4264 write 0x4a3c457 thru 0x4a41872 (0x541c bytes) 2024-09-13T09:18:42.008 INFO:teuthology.orchestra.run.smithi032.stdout:4265 trunc from 0x6c383c6 to 0xe95fad7 2024-09-13T09:18:42.018 INFO:teuthology.orchestra.run.smithi032.stdout:4267 read 0x84ecce9 thru 0x84ef6f5 (0x2a0d bytes) 2024-09-13T09:18:42.027 INFO:teuthology.orchestra.run.smithi032.stdout:4269 trunc from 0xe95fad7 to 0xaefc8ae 2024-09-13T09:18:42.045 INFO:teuthology.orchestra.run.smithi032.stdout:4270 read 0x7af4e89 thru 0x7aff843 (0xa9bb bytes) 2024-09-13T09:18:42.045 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.046 INFO:teuthology.orchestra.run.smithi032.stdout:4272 punch from 0x1b8166f to 0x1b84931, (0x32c2 bytes) 2024-09-13T09:18:42.053 INFO:teuthology.orchestra.run.smithi032.stdout:4273 trunc from 0xaefc8ae to 0xe0cc953 2024-09-13T09:18:42.065 INFO:teuthology.orchestra.run.smithi032.stdout:4274 writesame 0x6a024a7 thru 0x6a049f3 data_size 0x3(0x254d bytes) 2024-09-13T09:18:42.069 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:42.062+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:42.071 INFO:teuthology.orchestra.run.smithi032.stdout:4275 read 0x8333414 thru 0x833ff38 (0xcb25 bytes) 2024-09-13T09:18:42.072 INFO:teuthology.orchestra.run.smithi032.stdout:4277 write 0xbab7af2 thru 0xbac3ef5 (0xc404 bytes) 2024-09-13T09:18:42.075 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.076 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.076 INFO:teuthology.orchestra.run.smithi032.stdout:4282 read 0x9fa30dc thru 0x9fabc02 (0x8b27 bytes) 2024-09-13T09:18:42.076 INFO:teuthology.orchestra.run.smithi032.stdout:4283 write 0x9725729 thru 0x972c0d6 (0x69ae bytes) 2024-09-13T09:18:42.080 INFO:teuthology.orchestra.run.smithi032.stdout:4284 read 0x4efaa8e thru 0x4f0269a (0x7c0d bytes) 2024-09-13T09:18:42.081 INFO:teuthology.orchestra.run.smithi032.stdout:4285 write 0x196350a thru 0x19690bf (0x5bb6 bytes) 2024-09-13T09:18:42.085 INFO:teuthology.orchestra.run.smithi032.stdout:4286 write 0xb14dcd1 thru 0xb156ede (0x920e bytes) 2024-09-13T09:18:42.087 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:42.087 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:42.097 INFO:teuthology.orchestra.run.smithi032.stdout:4287 trunc from 0xe0cc953 to 0xa48ff50 2024-09-13T09:18:42.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:42.136 INFO:teuthology.orchestra.run.smithi032.stdout:4288 write 0xd2b8230 thru 0xd2bca6b (0x483c bytes) 2024-09-13T09:18:42.156 INFO:teuthology.orchestra.run.smithi032.stdout:4289 trunc from 0xd2bca6c to 0x68760d6 2024-09-13T09:18:42.170 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:42.166+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:42.173 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.190 INFO:teuthology.orchestra.run.smithi032.stdout:4294 write 0x845332a thru 0x845703b (0x3d12 bytes) 2024-09-13T09:18:42.193 INFO:teuthology.orchestra.run.smithi032.stdout:4297 read 0x279c7b thru 0x27f1b8 (0x553e bytes) 2024-09-13T09:18:42.194 INFO:teuthology.orchestra.run.smithi032.stdout:4299 write 0x715c407 thru 0x716787a (0xb474 bytes) 2024-09-13T09:18:42.215 INFO:teuthology.orchestra.run.smithi032.stdout:4300 write 0xab2b9a0 thru 0xab34196 (0x87f7 bytes) 2024-09-13T09:18:42.222 INFO:teuthology.orchestra.run.smithi032.stdout:4301 write 0xefc789 thru 0xf0340c (0x6c84 bytes) 2024-09-13T09:18:42.227 INFO:teuthology.orchestra.run.smithi032.stdout:4303 read 0x31e313 thru 0x32d1b2 (0xeea0 bytes) 2024-09-13T09:18:42.230 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-13T09:18:42.232 INFO:teuthology.orchestra.run.smithi032.stdout:4304 punch from 0x55cdd6c to 0x55d9b7c, (0xbe10 bytes) 2024-09-13T09:18:42.245 INFO:teuthology.orchestra.run.smithi032.stdout:4306 trunc from 0xab34197 to 0x27388dd 2024-09-13T09:18:42.271 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:42.266+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:42.286 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:42.286 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:42.287 INFO:teuthology.orchestra.run.smithi032.stdout:4307 write 0x5e72ad5 thru 0x5e734ef (0xa1b bytes) 2024-09-13T09:18:42.290 INFO:teuthology.orchestra.run.smithi032.stdout:4308 read 0x5a14ee9 thru 0x5a21bff (0xcd17 bytes) 2024-09-13T09:18:42.291 INFO:teuthology.orchestra.run.smithi032.stdout:4309 read 0x23eebc thru 0x242094 (0x31d9 bytes) 2024-09-13T09:18:42.292 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.301 INFO:teuthology.orchestra.run.smithi032.stdout:4311 trunc from 0x5e734f0 to 0xb1cf317 2024-09-13T09:18:42.305 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:42.312 INFO:teuthology.orchestra.run.smithi032.stdout:4312 read 0x76688df thru 0x7673e2b (0xb54d bytes) 2024-09-13T09:18:42.314 INFO:teuthology.orchestra.run.smithi032.stdout:4315 writesame 0xa00571e thru 0xa00c028 data_size 0x1(0x690b bytes) 2024-09-13T09:18:42.351 INFO:teuthology.orchestra.run.smithi032.stdout:4316 write 0x5fa3489 thru 0x5faddaf (0xa927 bytes) 2024-09-13T09:18:42.355 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.356 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.356 INFO:teuthology.orchestra.run.smithi032.stdout:4319 write 0x17f97d1 thru 0x180902d (0xf85d bytes) 2024-09-13T09:18:42.371 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:42.366+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:42.374 INFO:teuthology.orchestra.run.smithi032.stdout:4320 trunc from 0xb1cf317 to 0x19ad2b9 2024-09-13T09:18:42.409 INFO:teuthology.orchestra.run.smithi032.stdout:4321 trunc from 0x19ad2b9 to 0xb0e3b77 2024-09-13T09:18:42.420 INFO:teuthology.orchestra.run.smithi032.stdout:4323 read 0x5da376f thru 0x5daf95b (0xc1ed bytes) 2024-09-13T09:18:42.421 INFO:teuthology.orchestra.run.smithi032.stdout:4324 write 0x503036e thru 0x503078c (0x41f bytes) 2024-09-13T09:18:42.424 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.440 INFO:teuthology.orchestra.run.smithi032.stdout:4326 write 0xd196108 thru 0xd19654e (0x447 bytes) 2024-09-13T09:18:42.445 INFO:teuthology.orchestra.run.smithi032.stdout:4328 trunc from 0xd19654f to 0xe7a47cc 2024-09-13T09:18:42.449 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:42.449 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:42.465 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:42.472 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:42.466+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:42.484 INFO:teuthology.orchestra.run.smithi032.stdout:4329 trunc from 0xe7a47cc to 0x2a6531b 2024-09-13T09:18:42.502 INFO:teuthology.orchestra.run.smithi032.stdout:4330 punch from 0xc587d5 to 0xc602b3, (0x7ade bytes) 2024-09-13T09:18:42.502 INFO:teuthology.orchestra.run.smithi032.stdout:4331 read 0xa75c74 thru 0xa80a3c (0xadc9 bytes) 2024-09-13T09:18:42.530 INFO:teuthology.orchestra.run.smithi032.stdout:4332 write 0xc7c6008 thru 0xc7cc59a (0x6593 bytes) 2024-09-13T09:18:42.533 INFO:teuthology.orchestra.run.smithi032.stdout:4333 punch from 0xad3760f to 0xad39961, (0x2352 bytes) 2024-09-13T09:18:42.533 INFO:teuthology.orchestra.run.smithi032.stdout:4334 write 0x9dce996 thru 0x9dd25d3 (0x3c3e bytes) 2024-09-13T09:18:42.535 INFO:teuthology.orchestra.run.smithi032.stdout:4335 write 0x2a37edf thru 0x2a43588 (0xb6aa bytes) 2024-09-13T09:18:42.539 INFO:teuthology.orchestra.run.smithi032.stdout:4336 read 0x88fdbf7 thru 0x8908018 (0xa422 bytes) 2024-09-13T09:18:42.539 INFO:teuthology.orchestra.run.smithi032.stdout:4337 read 0x311cf48 thru 0x31228ba (0x5973 bytes) 2024-09-13T09:18:42.551 INFO:teuthology.orchestra.run.smithi032.stdout:4339 trunc from 0xc7cc59b to 0x533b35a 2024-09-13T09:18:42.573 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:42.570+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:42.594 INFO:teuthology.orchestra.run.smithi032.stdout:4340 write 0xd42413f thru 0xd42a900 (0x67c2 bytes) 2024-09-13T09:18:42.597 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.605 INFO:teuthology.orchestra.run.smithi032.stdout:4342 trunc from 0xd42a901 to 0x7d72f75 2024-09-13T09:18:42.624 INFO:teuthology.orchestra.run.smithi032.stdout:4343 read 0x4f86579 thru 0x4f907b8 (0xa240 bytes) 2024-09-13T09:18:42.624 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:42.625 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:42.625 INFO:teuthology.orchestra.run.smithi032.stdout:4344 punch from 0x40bd41b to 0x40bd5d6, (0x1bb bytes) 2024-09-13T09:18:42.625 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.625 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.627 INFO:teuthology.orchestra.run.smithi032.stdout:4348 trunc from 0x7d72f75 to 0x95280b4 2024-09-13T09:18:42.639 INFO:teuthology.orchestra.run.smithi032.stdout:4350 read 0x2acc160 thru 0x2acf6eb (0x358c bytes) 2024-09-13T09:18:42.640 INFO:teuthology.orchestra.run.smithi032.stdout:4351 write 0x1f582f5 thru 0x1f6236b (0xa077 bytes) 2024-09-13T09:18:42.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:42.644 INFO:teuthology.orchestra.run.smithi032.stdout:4352 writesame 0x3aedcc thru 0x3bd843 data_size 0x3d(0xea78 bytes) 2024-09-13T09:18:42.648 INFO:teuthology.orchestra.run.smithi032.stdout:4353 trunc from 0x95280b4 to 0xa700487 2024-09-13T09:18:42.667 INFO:teuthology.orchestra.run.smithi032.stdout:4354 trunc from 0xa700487 to 0x5b79342 2024-09-13T09:18:42.674 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:42.670+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:42.684 INFO:teuthology.orchestra.run.smithi032.stdout:4355 writesame 0x4f082f6 thru 0x4f146fa data_size 0x2b(0xc405 bytes) 2024-09-13T09:18:42.704 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:18:42.704 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:18:42.705 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-13T09:18:42.705 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:18:42.705 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:18:42.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-13T09:18:42.708 INFO:teuthology.orchestra.run.smithi032.stdout:4356 writesame 0x99fb85d thru 0x9a0900b data_size 0x2d(0xd7af bytes) 2024-09-13T09:18:42.730 INFO:teuthology.orchestra.run.smithi032.stdout:4357 write 0xecb60a3 thru 0xecbce86 (0x6de4 bytes) 2024-09-13T09:18:42.737 INFO:teuthology.orchestra.run.smithi032.stdout:4358 trunc from 0xecbce87 to 0xb9ef7d8 2024-09-13T09:18:42.766 INFO:teuthology.orchestra.run.smithi032.stdout:4359 writesame 0xc3ca52d thru 0xc3ccee4 data_size 0xa(0x29b8 bytes) 2024-09-13T09:18:42.768 INFO:teuthology.orchestra.run.smithi032.stdout:4360 write 0xc84d2e thru 0xc84ff1 (0x2c4 bytes) 2024-09-13T09:18:42.774 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:42.774+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:42.781 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:42.782 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:42.786 INFO:teuthology.orchestra.run.smithi032.stdout:4361 write 0xcb79953 thru 0xcb7eadc (0x518a bytes) 2024-09-13T09:18:42.788 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.789 INFO:teuthology.orchestra.run.smithi032.stdout:4363 read 0xaefb20a thru 0xaefb619 (0x410 bytes) 2024-09-13T09:18:42.789 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:42.794 INFO:teuthology.orchestra.run.smithi032.stdout:4366 trunc from 0xcb7eadd to 0x9b21824 2024-09-13T09:18:42.803 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:42.810 INFO:teuthology.orchestra.run.smithi032.stdout:4370 read 0x1ed8adb thru 0x1edb5ac (0x2ad2 bytes) 2024-09-13T09:18:42.811 INFO:teuthology.orchestra.run.smithi032.stdout:4371 write 0x2ccfc2a thru 0x2cdf5a2 (0xf979 bytes) 2024-09-13T09:18:42.825 INFO:teuthology.orchestra.run.smithi032.stdout:4372 writesame 0x9de6476 thru 0x9df0b2a data_size 0x1(0xa6b5 bytes) 2024-09-13T09:18:42.835 INFO:teuthology.orchestra.run.smithi032.stdout:4374 trunc from 0x9df0b2b to 0xebb4380 2024-09-13T09:18:42.845 INFO:teuthology.orchestra.run.smithi032.stdout:4375 write 0x273b690 thru 0x274aaa1 (0xf412 bytes) 2024-09-13T09:18:42.846 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:18:42.846 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:42.847 INFO:teuthology.orchestra.run.smithi032.stdout:4376 clone 31 order 23 su 262144 sc 8 2024-09-13T09:18:42.875 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:42.870+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:42.933 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:42.933 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:42.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:42.995 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:42.994+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:43.073 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:43.074 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:43.077 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:43.070+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:43.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:43.178 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:43.178+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:43.256 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:43.256 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:43.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:43.279 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:43.274+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:43.381 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:43.374+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:43.400 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:43.400 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:43.406 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone30 from 0xebb4380 (overlap 0x184d8ba) to 0x15df641 2024-09-13T09:18:43.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:43.482 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:43.478+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:43.545 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:43.603 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:43.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:43.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:43.606 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:43.602+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:43.639 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-13T09:18:43.706 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:43.702+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:43.779 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:43.779 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:43.779 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:43.779 INFO:teuthology.orchestra.run.smithi032.stdout:4383 read 0xa6f5dea thru 0xa704b95 (0xedac bytes) 2024-09-13T09:18:43.781 INFO:teuthology.orchestra.run.smithi032.stdout:4386 punch from 0x54e324e to 0x54f0802, (0xd5b4 bytes) 2024-09-13T09:18:43.784 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:43.784 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:43.789 INFO:teuthology.orchestra.run.smithi032.stdout:4387 read 0xca4d5 thru 0xd1686 (0x71b2 bytes) 2024-09-13T09:18:43.790 INFO:teuthology.orchestra.run.smithi032.stdout:4388 punch from 0x9a66834 to 0x9a70417, (0x9be3 bytes) 2024-09-13T09:18:43.790 INFO:teuthology.orchestra.run.smithi032.stdout:4390 writesame 0x2dd421d thru 0x2de407b data_size 0x1(0xfe5f bytes) 2024-09-13T09:18:43.794 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:43.794 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:43.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:43.807 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:43.802+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:43.811 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:43.828 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:43.845 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:43.846 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:43.877 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:43.878 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:43.885 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:43.885 INFO:teuthology.orchestra.run.smithi032.stdout:4393 read 0x227adca thru 0x2282f21 (0x8158 bytes) 2024-09-13T09:18:43.885 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:43.885 INFO:teuthology.orchestra.run.smithi032.stdout:4395 read 0xcd26655 thru 0xcd2bf0b (0x58b7 bytes) 2024-09-13T09:18:43.889 INFO:teuthology.orchestra.run.smithi032.stdout:4396 trunc from 0xebb4380 to 0xc6e8f5a 2024-09-13T09:18:43.908 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:43.902+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:43.909 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:43.909 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:43.935 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:43.938 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:43.938 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:18:43.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:18:43.938 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:18:43.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.938 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:43.938 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:43.939 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.399402254000002, 2024-09-13T09:18:43.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:18:43.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.939 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:43.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.939 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:18:43.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.939 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:18:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:18:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:18:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:43.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:18:43.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:18:43.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:43.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:18:43.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:18:43.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:43.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:18:43.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:18:43.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:43.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:18:43.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:18:43.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:43.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:18:43.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:18:43.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:43.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:18:43.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:18:43.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.945 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:43.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.946 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:43.946 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:43.946 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029835268999999, 2024-09-13T09:18:43.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:18:43.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.946 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:43.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:18:43.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.947 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:43.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:18:43.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:43.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:18:43.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:43.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:18:43.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:43.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029817948000002, 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.951 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:18:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:18:43.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:18:43.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029801007, 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.956 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:18:43.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.956 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:43.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:18:43.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:43.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:18:43.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:43.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:43.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:18:43.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:18:43.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:43.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:18:43.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:18:43.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:43.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:18:43.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:18:43.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:43.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:18:43.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:18:43.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:43.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:18:43.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:18:43.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.960 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:43.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.961 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:43.961 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:43.961 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029783778999999, 2024-09-13T09:18:43.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:18:43.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:43.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:18:43.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.962 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:43.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:43.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:43.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:18:43.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:18:43.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:43.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:18:43.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:18:43.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:43.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:18:43.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:18:43.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:43.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:18:43.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:18:43.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.965 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:43.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.965 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029747358000002, 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:43.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:43.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:43.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:18:43.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:43.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:43.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:18:43.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:18:43.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:43.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:18:43.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029693896000001, 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:43.970 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:43.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:18:43.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:43.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:43.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:18:43.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:18:43.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:43.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:18:43.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:18:43.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:43.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:18:43.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:18:43.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:43.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:18:43.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:18:43.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:43.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:18:43.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:18:43.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.974 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:43.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029671875999998, 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.975 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:18:43.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:43.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:43.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:18:43.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:18:43.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:43.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:18:43.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:18:43.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:43.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:18:43.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:18:43.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:43.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:18:43.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:18:43.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029647240999999, 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:43.979 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:18:43.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:43.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:18:43.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:43.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:43.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:18:43.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:18:43.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:18:43.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029628718000001, 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.983 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:43.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.984 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:18:43.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.984 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:43.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:43.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:43.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:18:43.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:43.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:43.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.988 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:43.988 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:43.988 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029610847000001, 2024-09-13T09:18:43.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:18:43.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.988 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:43.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.988 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:18:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:43.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:18:43.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:43.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:43.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:18:43.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:18:43.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:43.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:18:43.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:18:43.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:43.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:18:43.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:18:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:18:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:18:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:18:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:18:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029593516999999, 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:18:43.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:43.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:18:43.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:18:43.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:43.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:18:43.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:18:43.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:43.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:18:43.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:18:43.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.997 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:43.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:43.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.997 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:43.997 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:43.997 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029576046999999, 2024-09-13T09:18:43.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:18:43.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:43.998 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:43.998 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:43.998 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:43.998 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:43.998 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:18:43.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.998 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:43.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:43.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:43.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:43.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:18:43.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:43.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:43.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:43.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:18:44.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:18:44.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:44.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:18:44.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:18:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:18:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:18:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029559181, 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029541393999999, 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:44.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:44.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:44.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:44.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:44.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:44.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:18:44.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:44.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:44.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:44.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:18:44.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:18:44.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:44.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:18:44.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:18:44.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.012 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:44.012 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.012 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:44.012 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:44.012 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.029497083999999, 2024-09-13T09:18:44.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:18:44.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:44.013 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:18:44.013 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:44.013 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:44.013 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.013 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:18:44.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.013 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:44.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:44.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:44.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:44.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:18:44.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:18:44.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:44.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:18:44.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:18:44.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:18:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:18:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:18:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:18:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:18:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:18:44.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.017 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:44.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.017 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:44.017 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:44.017 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.796366595999999, 2024-09-13T09:18:44.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:18:44.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:44.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:44.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:44.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:44.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.018 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:18:44.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.018 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:44.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:44.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:44.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:44.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:44.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:44.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:18:44.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:44.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:18:44.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:44.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:18:44.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:18:44.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.022 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:44.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.022 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:44.022 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:44.022 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.796349762999998, 2024-09-13T09:18:44.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:18:44.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:44.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:44.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.796333404999999, 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.027 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:18:44.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:18:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:18:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:18:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:18:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:18:44.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:18:44.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:44.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:18:44.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:18:44.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:44.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:18:44.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:18:44.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:44.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:18:44.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:18:44.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.031 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:44.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.031 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:66592 3.7 3:e0817996:::rbd_data.1185b1e141e2.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:18:44.032 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:18:44.032 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2919736750000004, 2024-09-13T09:18:44.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.051168777999999998, 2024-09-13T09:18:44.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:18:44.032 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:44.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:18:44.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:18:44.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.032 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 66592 2024-09-13T09:18:44.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.032 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:18:44.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:18:44.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:18:44.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:18:44.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:18:44.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:18:44.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:18:44.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643787+0000", 2024-09-13T09:18:44.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:44.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:18:44.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643799+0000", 2024-09-13T09:18:44.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2140000000000001e-05 2024-09-13T09:18:44.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643827+0000", 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8121999999999999e-05 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643841+0000", 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3969e-05 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643985+0000", 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014404199999999999 2024-09-13T09:18:44.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.644371+0000", 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038588700000000001 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.654317+0000", 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099452050000000004 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694641+0000", 2024-09-13T09:18:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.040324802999999999 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694682+0000", 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0451999999999998e-05 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694828+0000", 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014580499999999999 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:18:44.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694958+0000", 2024-09-13T09:18:44.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013029599999999999 2024-09-13T09:18:44.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.038 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:44.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:18:44.038 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:18:44.038 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:44.039 INFO:teuthology.orchestra.run.smithi032.stdout:4397 punch from 0xa817d1 to 0xa906f8, (0xef27 bytes) 2024-09-13T09:18:44.039 INFO:teuthology.orchestra.run.smithi032.stdout:4399 read 0x8b26476 thru 0x8b2a00f (0x3b9a bytes) 2024-09-13T09:18:44.039 INFO:teuthology.orchestra.run.smithi032.stdout:4400 read 0x7b831e2 thru 0x7b91d6b (0xeb8a bytes) 2024-09-13T09:18:44.039 INFO:teuthology.orchestra.run.smithi032.stdout:4401 punch from 0x1f8beab to 0x1f8df7f, (0x20d4 bytes) 2024-09-13T09:18:44.040 INFO:teuthology.orchestra.run.smithi032.stdout:4402 write 0x8e2a520 thru 0x8e3a037 (0xfb18 bytes) 2024-09-13T09:18:44.040 INFO:teuthology.orchestra.run.smithi032.stdout:4405 write 0x82aca62 thru 0x82b1ae0 (0x507f bytes) 2024-09-13T09:18:44.040 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:44.041 INFO:teuthology.orchestra.run.smithi032.stdout:4406 read 0x781d1b3 thru 0x7820ddf (0x3c2d bytes) 2024-09-13T09:18:44.041 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.041 INFO:teuthology.orchestra.run.smithi032.stdout:4411 writesame 0x405df1d thru 0x405f58c data_size 0x8(0x1670 bytes) 2024-09-13T09:18:44.041 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:44.042 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.042 INFO:teuthology.orchestra.run.smithi032.stdout:4413 write 0xbea2d57 thru 0xbeae087 (0xb331 bytes) 2024-09-13T09:18:44.043 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.043 INFO:teuthology.orchestra.run.smithi032.stdout:4417 read 0xb86ff51 thru 0xb87baee (0xbb9e bytes) 2024-09-13T09:18:44.043 INFO:teuthology.orchestra.run.smithi032.stdout:4418 writesame 0x29c27e9 thru 0x29c3dcf data_size 0x15(0x15e7 bytes) 2024-09-13T09:18:44.043 INFO:teuthology.orchestra.run.smithi032.stdout:4419 write 0xab6f05c thru 0xab742a9 (0x524e bytes) 2024-09-13T09:18:44.044 INFO:teuthology.orchestra.run.smithi032.stdout:4422 trunc from 0xc6e8f5a to 0xd30408d 2024-09-13T09:18:44.044 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.044 INFO:teuthology.orchestra.run.smithi032.stdout:4426 writesame 0xb63d9b2 thru 0xb648619 data_size 0x24(0xac68 bytes) 2024-09-13T09:18:44.044 INFO:teuthology.orchestra.run.smithi032.stdout:4428 trunc from 0xd30408d to 0x647faa9 2024-09-13T09:18:44.045 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:44.034+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:44.089 INFO:teuthology.orchestra.run.smithi032.stdout:4430 writesame 0x22564ab thru 0x225e211 data_size 0x57(0x7d67 bytes) 2024-09-13T09:18:44.107 INFO:teuthology.orchestra.run.smithi032.stdout:4431 write 0x23a616 thru 0x24262a (0x8015 bytes) 2024-09-13T09:18:44.140 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:44.138+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:44.154 INFO:teuthology.orchestra.run.smithi032.stdout:4433 write 0xa36f16f thru 0xa375fca (0x6e5c bytes) 2024-09-13T09:18:44.181 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:44.181 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:44.181 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:44.181 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:44.184 INFO:teuthology.orchestra.run.smithi032.stdout:4436 trunc from 0xa375fcb to 0x28fed65 2024-09-13T09:18:44.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:44.241 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:44.238+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:44.250 INFO:teuthology.orchestra.run.smithi032.stdout:4437 trunc from 0x28fed65 to 0x1b42ae7 2024-09-13T09:18:44.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:18:44.266 INFO:teuthology.orchestra.run.smithi032.stdout:4438 punch from 0x1936246 to 0x1938c3b, (0x29f5 bytes) 2024-09-13T09:18:44.276 INFO:teuthology.orchestra.run.smithi032.stdout:4439 trunc from 0x1b42ae7 to 0x579b623 2024-09-13T09:18:44.309 INFO:teuthology.orchestra.run.smithi032.stdout:4440 write 0xba9be57 thru 0xbaa3121 (0x72cb bytes) 2024-09-13T09:18:44.329 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:44.330 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:44.336 INFO:teuthology.orchestra.run.smithi032.stdout:4441 trunc from 0xbaa3122 to 0xc98a48 2024-09-13T09:18:44.341 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:44.338+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:44.345 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:44.383 INFO:teuthology.orchestra.run.smithi032.stdout:4442 write 0x9f385dc thru 0x9f41ec4 (0x98e9 bytes) 2024-09-13T09:18:44.391 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.402 INFO:teuthology.orchestra.run.smithi032.stdout:4444 write 0xa55be4b thru 0xa560cb7 (0x4e6d bytes) 2024-09-13T09:18:44.404 INFO:teuthology.orchestra.run.smithi032.stdout:4445 write 0x3dc3d8c thru 0x3dc91e2 (0x5457 bytes) 2024-09-13T09:18:44.407 INFO:teuthology.orchestra.run.smithi032.stdout:4446 read 0x847155d thru 0x8475055 (0x3af9 bytes) 2024-09-13T09:18:44.407 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.407 INFO:teuthology.orchestra.run.smithi032.stdout:4449 writesame 0x8052189 thru 0x8053a3c data_size 0x6(0x18b4 bytes) 2024-09-13T09:18:44.413 INFO:teuthology.orchestra.run.smithi032.stdout:4451 writesame 0x8ac979c thru 0x8ad4972 data_size 0x1(0xb1d7 bytes) 2024-09-13T09:18:44.442 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:44.438+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:44.468 INFO:teuthology.orchestra.run.smithi032.stdout:4452 trunc from 0xa560cb8 to 0x1385164 2024-09-13T09:18:44.497 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.528 INFO:teuthology.orchestra.run.smithi032.stdout:4454 write 0xaa077b7 thru 0xaa16b8a (0xf3d4 bytes) 2024-09-13T09:18:44.531 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4388 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.542682356, 2024-09-13T09:18:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069349509000000004, 2024-09-13T09:18:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4388 2024-09-13T09:18:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985769+0000", 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985775+0000", 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5230000000000001e-06 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985794+0000", 2024-09-13T09:18:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8425999999999999e-05 2024-09-13T09:18:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985809+0000", 2024-09-13T09:18:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.486e-05 2024-09-13T09:18:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.054965+0000", 2024-09-13T09:18:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069156463000000001 2024-09-13T09:18:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055036+0000", 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0780999999999999e-05 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055120+0000", 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3486000000000007e-05 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4389 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.542631833000002, 2024-09-13T09:18:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069473833999999998, 2024-09-13T09:18:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4389 2024-09-13T09:18:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985820+0000", 2024-09-13T09:18:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985859+0000", 2024-09-13T09:18:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9279999999999997e-05 2024-09-13T09:18:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985881+0000", 2024-09-13T09:18:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2116999999999998e-05 2024-09-13T09:18:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985896+0000", 2024-09-13T09:18:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4698999999999999e-05 2024-09-13T09:18:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055152+0000", 2024-09-13T09:18:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069256568000000004 2024-09-13T09:18:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055220+0000", 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8255000000000003e-05 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055294+0000", 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3858999999999999e-05 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4390 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.542536220999999, 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069563635999999998, 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4390 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985915+0000", 2024-09-13T09:18:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985922+0000", 2024-09-13T09:18:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:18:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985941+0000", 2024-09-13T09:18:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8260000000000001e-05 2024-09-13T09:18:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985955+0000", 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4409999999999999e-05 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055325+0000", 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069370290000000001 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055395+0000", 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0158000000000002e-05 2024-09-13T09:18:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055480+0000", 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4340999999999998e-05 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.542485153999998, 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:18:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:18:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:18:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:18:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:18:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:18:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.542436418000001, 2024-09-13T09:18:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:18:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:18:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:18:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:18:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:18:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:18:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:18:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:18:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:18:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:18:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.542368748999998, 2024-09-13T09:18:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:18:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:18:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:18:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:18:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:18:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:18:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:18:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:18:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.542228539999996, 2024-09-13T09:18:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:18:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:18:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:18:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:18:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.542137674000003, 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:18:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.542062813999998, 2024-09-13T09:18:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:18:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.541991637999999, 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:18:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:18:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:18:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.541942585000001, 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:18:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.541880849000002, 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:18:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:18:44.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:18:44.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:18:44.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:18:44.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:18:44.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.541831178999999, 2024-09-13T09:18:44.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.592 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.541782660000003, 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:18:44.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.541734263999999, 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.541686976999998, 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.541603575000003, 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.541529849, 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:18:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.541461726000001, 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:18:44.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:18:44.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.541411259, 2024-09-13T09:18:44.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:44.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:44.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.622 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:44.623 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:44.623 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:44.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:44.625 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:44.618+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:44.626 INFO:teuthology.orchestra.run.smithi032.stdout:4456 writesame 0x9d04f4a thru 0x9d0db05 data_size 0x42(0x8bbc bytes) 2024-09-13T09:18:44.627 INFO:teuthology.orchestra.run.smithi032.stdout:4457 punch from 0x876aead to 0x8778157, (0xd2aa bytes) 2024-09-13T09:18:44.627 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.627 INFO:teuthology.orchestra.run.smithi032.stdout:4463 write 0x62a74a1 thru 0x62b5e41 (0xe9a1 bytes) 2024-09-13T09:18:44.627 INFO:teuthology.orchestra.run.smithi032.stdout:4464 punch from 0x2015fbb to 0x201f9f5, (0x9a3a bytes) 2024-09-13T09:18:44.627 INFO:teuthology.orchestra.run.smithi032.stdout:4466 write 0x8527dfb thru 0x852e99d (0x6ba3 bytes) 2024-09-13T09:18:44.627 INFO:teuthology.orchestra.run.smithi032.stdout:4467 write 0x5b17eee thru 0x5b2462a (0xc73d bytes) 2024-09-13T09:18:44.627 INFO:teuthology.orchestra.run.smithi032.stdout:4468 write 0x8d688ab thru 0x8d72bc8 (0xa31e bytes) 2024-09-13T09:18:44.627 INFO:teuthology.orchestra.run.smithi032.stdout:4469 read 0xa1b6253 thru 0xa1b8bba (0x2968 bytes) 2024-09-13T09:18:44.627 INFO:teuthology.orchestra.run.smithi032.stdout:4471 punch from 0x9d3ff08 to 0x9d41b34, (0x1c2c bytes) 2024-09-13T09:18:44.627 INFO:teuthology.orchestra.run.smithi032.stdout:4472 trunc from 0xaa16b8b to 0x834dcd9 2024-09-13T09:18:44.627 INFO:teuthology.orchestra.run.smithi032.stdout:4473 write 0x60e4995 thru 0x60f09ec (0xc058 bytes) 2024-09-13T09:18:44.627 INFO:teuthology.orchestra.run.smithi032.stdout:4474 read 0x38cd809 thru 0x38dbcf0 (0xe4e8 bytes) 2024-09-13T09:18:44.628 INFO:teuthology.orchestra.run.smithi032.stdout:4475 write 0x1acd305 thru 0x1ad5f02 (0x8bfe bytes) 2024-09-13T09:18:44.628 INFO:teuthology.orchestra.run.smithi032.stdout:4476 write 0xcaa67eb thru 0xcab63e1 (0xfbf7 bytes) 2024-09-13T09:18:44.628 INFO:teuthology.orchestra.run.smithi032.stdout:4478 write 0xa9fd7c8 thru 0xaa0a8b6 (0xd0ef bytes) 2024-09-13T09:18:44.628 INFO:teuthology.orchestra.run.smithi032.stdout:4479 read 0x1a0b61 thru 0x1afb4f (0xefef bytes) 2024-09-13T09:18:44.628 INFO:teuthology.orchestra.run.smithi032.stdout:4480 write 0xb4f8f9c thru 0xb50785a (0xe8bf bytes) 2024-09-13T09:18:44.634 INFO:teuthology.orchestra.run.smithi032.stdout:4481 writesame 0x7b4ab0c thru 0x7b598bb data_size 0x8(0xedb0 bytes) 2024-09-13T09:18:44.646 INFO:teuthology.orchestra.run.smithi032.stdout:4482 write 0xca2092c thru 0xca29ab4 (0x9189 bytes) 2024-09-13T09:18:44.654 INFO:teuthology.orchestra.run.smithi032.stdout:4483 read 0x1c8d265 thru 0x1c94aba (0x7856 bytes) 2024-09-13T09:18:44.655 INFO:teuthology.orchestra.run.smithi032.stdout:4484 write 0x28c52ad thru 0x28cc7a0 (0x74f4 bytes) 2024-09-13T09:18:44.659 INFO:teuthology.orchestra.run.smithi032.stdout:4485 read 0x3e6b898 thru 0x3e6d2af (0x1a18 bytes) 2024-09-13T09:18:44.660 INFO:teuthology.orchestra.run.smithi032.stdout:4486 write 0xa3b579b thru 0xa3b5a3d (0x2a3 bytes) 2024-09-13T09:18:44.667 INFO:teuthology.orchestra.run.smithi032.stdout:4487 read 0x347a1df thru 0x347ee1c (0x4c3e bytes) 2024-09-13T09:18:44.668 INFO:teuthology.orchestra.run.smithi032.stdout:4489 write 0x2b678c4 thru 0x2b77690 (0xfdcd bytes) 2024-09-13T09:18:44.673 INFO:teuthology.orchestra.run.smithi032.stdout:4491 write 0x22c8582 thru 0x22d41e3 (0xbc62 bytes) 2024-09-13T09:18:44.677 INFO:teuthology.orchestra.run.smithi032.stdout:4493 write 0x1a30c80 thru 0x1a3b03c (0xa3bd bytes) 2024-09-13T09:18:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:4495 trunc from 0xcab63e2 to 0x232df49 2024-09-13T09:18:44.725 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:44.718+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:4496 trunc from 0x232df49 to 0xb0cc8f2 2024-09-13T09:18:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:4498 trunc from 0xb0cc8f2 to 0x50af4db 2024-09-13T09:18:44.797 INFO:teuthology.orchestra.run.smithi032.stdout:4499 read 0x47401cb thru 0x4745e58 (0x5c8e bytes) 2024-09-13T09:18:44.807 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:44.807 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:44.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:4501 writesame 0xdf0582b thru 0xdf07ad2 data_size 0x4(0x22a8 bytes) 2024-09-13T09:18:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:4506 write 0x591fd47 thru 0x592c3d6 (0xc690 bytes) 2024-09-13T09:18:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:4507 write 0x37143fd thru 0x371cd5d (0x8961 bytes) 2024-09-13T09:18:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:4511 write 0x25b86e4 thru 0x25c7d88 (0xf6a5 bytes) 2024-09-13T09:18:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:4512 writesame 0xd62aba thru 0xd63951 data_size 0x4(0xe98 bytes) 2024-09-13T09:18:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:4513 writesame 0x9bd7315 thru 0x9be3101 data_size 0x13(0xbded bytes) 2024-09-13T09:18:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:4515 read 0x8c4e5d3 thru 0x8c510ab (0x2ad9 bytes) 2024-09-13T09:18:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:4516 writesame 0x83e5f7a thru 0x83ed090 data_size 0x11(0x7117 bytes) 2024-09-13T09:18:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:4518 read 0x245b253 thru 0x24698f6 (0xe6a4 bytes) 2024-09-13T09:18:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:4519 write 0x6159545 thru 0x6160ac5 (0x7581 bytes) 2024-09-13T09:18:44.865 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:44.862+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:4520 punch from 0x2621c52 to 0x2629d66, (0x8114 bytes) 2024-09-13T09:18:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:4523 read 0x47189e2 thru 0x471cde4 (0x4403 bytes) 2024-09-13T09:18:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:4525 write 0x4a5c1aa thru 0x4a60a43 (0x489a bytes) 2024-09-13T09:18:44.879 INFO:teuthology.orchestra.run.smithi032.stdout:4526 read 0x80d61a8 thru 0x80d6406 (0x25f bytes) 2024-09-13T09:18:44.880 INFO:teuthology.orchestra.run.smithi032.stdout:4527 write 0x6a81566 thru 0x6a84a76 (0x3511 bytes) 2024-09-13T09:18:44.886 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.886 INFO:teuthology.orchestra.run.smithi032.stdout:4532 writesame 0xc04abf9 thru 0xc04bc31 data_size 0x1(0x1039 bytes) 2024-09-13T09:18:44.893 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.893 INFO:teuthology.orchestra.run.smithi032.stdout:4534 punch from 0x9bac05d to 0x9bb60bf, (0xa062 bytes) 2024-09-13T09:18:44.893 INFO:teuthology.orchestra.run.smithi032.stdout:4535 write 0xa25155b thru 0xa2587a7 (0x724d bytes) 2024-09-13T09:18:44.898 INFO:teuthology.orchestra.run.smithi032.stdout:4536 read 0xd891f46 thru 0xd897f6b (0x6026 bytes) 2024-09-13T09:18:44.899 INFO:teuthology.orchestra.run.smithi032.stdout:4538 writesame 0x32545ef thru 0x325a9e3 data_size 0x1(0x63f5 bytes) 2024-09-13T09:18:44.927 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:44.926+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:44.933 INFO:teuthology.orchestra.run.smithi032.stdout:4539 read 0xcf78c20 thru 0xcf799a7 (0xd88 bytes) 2024-09-13T09:18:44.933 INFO:teuthology.orchestra.run.smithi032.stdout:4540 read 0xde1009a thru 0xde14e9a (0x4e01 bytes) 2024-09-13T09:18:44.933 INFO:teuthology.orchestra.run.smithi032.stdout:4541 write 0x161fc5c thru 0x16271ea (0x758f bytes) 2024-09-13T09:18:44.937 INFO:teuthology.orchestra.run.smithi032.stdout:4544 read 0x419081a thru 0x419f109 (0xe8f0 bytes) 2024-09-13T09:18:44.938 INFO:teuthology.orchestra.run.smithi032.stdout:4545 trunc from 0xdf07ad3 to 0xe47e7b7 2024-09-13T09:18:44.951 INFO:teuthology.orchestra.run.smithi032.stdout:4547 read 0xd983dcf thru 0xd99124e (0xd480 bytes) 2024-09-13T09:18:44.951 INFO:teuthology.orchestra.run.smithi032.stdout:4549 read 0xe31ec93 thru 0xe32b006 (0xc374 bytes) 2024-09-13T09:18:44.951 INFO:teuthology.orchestra.run.smithi032.stdout:4550 write 0xb061f1 thru 0xb06f33 (0xd43 bytes) 2024-09-13T09:18:44.955 INFO:teuthology.orchestra.run.smithi032.stdout:4551 writesame 0x96be38c thru 0x96cc6e9 data_size 0x6(0xe35e bytes) 2024-09-13T09:18:44.973 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.973 INFO:teuthology.orchestra.run.smithi032.stdout:4556 write 0xa9d9e59 thru 0xa9dcfe5 (0x318d bytes) 2024-09-13T09:18:44.976 INFO:teuthology.orchestra.run.smithi032.stdout:4558 writesame 0x31a5c9e thru 0x31a8241 data_size 0x42(0x25a4 bytes) 2024-09-13T09:18:44.980 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:44.980 INFO:teuthology.orchestra.run.smithi032.stdout:4561 read 0x9469af5 thru 0x9470518 (0x6a24 bytes) 2024-09-13T09:18:44.981 INFO:teuthology.orchestra.run.smithi032.stdout:4562 write 0xc0f277 thru 0xc15f78 (0x6d02 bytes) 2024-09-13T09:18:44.985 INFO:teuthology.orchestra.run.smithi032.stdout:4563 read 0x784e700 thru 0x7851db9 (0x36ba bytes) 2024-09-13T09:18:44.986 INFO:teuthology.orchestra.run.smithi032.stdout:4564 punch from 0x1369b11 to 0x1378650, (0xeb3f bytes) 2024-09-13T09:18:44.987 INFO:teuthology.orchestra.run.smithi032.stdout:4565 read 0x38039f9 thru 0x38134fe (0xfb06 bytes) 2024-09-13T09:18:44.987 INFO:teuthology.orchestra.run.smithi032.stdout:4566 write 0x6e69dec thru 0x6e6efde (0x51f3 bytes) 2024-09-13T09:18:44.991 INFO:teuthology.orchestra.run.smithi032.stdout:4567 write 0x90a0fc3 thru 0x90a3c92 (0x2cd0 bytes) 2024-09-13T09:18:44.995 INFO:teuthology.orchestra.run.smithi032.stdout:4570 writesame 0x79c972e thru 0x79cfa13 data_size 0x1(0x62e6 bytes) 2024-09-13T09:18:45.003 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:45.003 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:45.003 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:45.003 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:45.003 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:45.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:45.034 INFO:teuthology.orchestra.run.smithi032.stdout:4573 read 0x8ea06c5 thru 0x8eade1c (0xd758 bytes) 2024-09-13T09:18:45.035 INFO:teuthology.orchestra.run.smithi032.stdout:4574 punch from 0x7988cd9 to 0x798c8de, (0x3c05 bytes) 2024-09-13T09:18:45.035 INFO:teuthology.orchestra.run.smithi032.stdout:4575 punch from 0xe3462f3 to 0xe34de96, (0x7ba3 bytes) 2024-09-13T09:18:45.035 INFO:teuthology.orchestra.run.smithi032.stdout:4577 read 0x9d46c42 thru 0x9d56048 (0xf407 bytes) 2024-09-13T09:18:45.035 INFO:teuthology.orchestra.run.smithi032.stdout:4580 write 0xba525ff thru 0xba569fc (0x43fe bytes) 2024-09-13T09:18:45.039 INFO:teuthology.orchestra.run.smithi032.stdout:4581 punch from 0x40cb95b to 0x40cf0d2, (0x3777 bytes) 2024-09-13T09:18:45.039 INFO:teuthology.orchestra.run.smithi032.stdout:4582 write 0xa6f4cbb thru 0xa6fbf8a (0x72d0 bytes) 2024-09-13T09:18:45.042 INFO:teuthology.orchestra.run.smithi032.stdout:4583 read 0xab98dc7 thru 0xaba0714 (0x794e bytes) 2024-09-13T09:18:45.042 INFO:teuthology.orchestra.run.smithi032.stdout:4584 punch from 0x445527a to 0x4458f6f, (0x3cf5 bytes) 2024-09-13T09:18:45.042 INFO:teuthology.orchestra.run.smithi032.stdout:4585 read 0x748dad8 thru 0x749d44d (0xf976 bytes) 2024-09-13T09:18:45.043 INFO:teuthology.orchestra.run.smithi032.stdout:4587 write 0x348f942 thru 0x348ff50 (0x60f bytes) 2024-09-13T09:18:45.045 INFO:teuthology.orchestra.run.smithi032.stdout:4588 write 0xcdbdd6 thru 0xcddbb3 (0x1dde bytes) 2024-09-13T09:18:45.049 INFO:teuthology.orchestra.run.smithi032.stdout:4589 write 0x89b80e0 thru 0x89be656 (0x6577 bytes) 2024-09-13T09:18:45.054 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:45.055 INFO:teuthology.orchestra.run.smithi032.stdout:4591 punch from 0x1409cf8 to 0x140d5f7, (0x38ff bytes) 2024-09-13T09:18:45.055 INFO:teuthology.orchestra.run.smithi032.stdout:4592 write 0xd1d2718 thru 0xd1d9499 (0x6d82 bytes) 2024-09-13T09:18:45.060 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:45.060 INFO:teuthology.orchestra.run.smithi032.stdout:4594 read 0x7c5b20a thru 0x7c64c31 (0x9a28 bytes) 2024-09-13T09:18:45.061 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:45.061 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:45.061 INFO:teuthology.orchestra.run.smithi032.stdout:4598 punch from 0xc0e863e to 0xc0f7863, (0xf225 bytes) 2024-09-13T09:18:45.061 INFO:teuthology.orchestra.run.smithi032.stdout:4599 read 0x94d551b thru 0x94e2cdb (0xd7c1 bytes) 2024-09-13T09:18:45.061 INFO:teuthology.orchestra.run.smithi032.stdout:4600 punch from 0x7d32504 to 0x7d34be1, (0x26dd bytes) 2024-09-13T09:18:45.061 INFO:teuthology.orchestra.run.smithi032.stdout:4601 writesame 0xc21be10 thru 0xc222acc data_size 0x9(0x6cbd bytes) 2024-09-13T09:18:45.065 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:45.062+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:45.067 INFO:teuthology.orchestra.run.smithi032.stdout:4603 writesame 0x4270fff thru 0x42807fc data_size 0x9(0xf7fe bytes) 2024-09-13T09:18:45.077 INFO:teuthology.orchestra.run.smithi032.stdout:4607 writesame 0x5923b0e thru 0x592b02e data_size 0x3(0x7521 bytes) 2024-09-13T09:18:45.085 INFO:teuthology.orchestra.run.smithi032.stdout:4609 punch from 0x2a985ae to 0x2a990de, (0xb30 bytes) 2024-09-13T09:18:45.085 INFO:teuthology.orchestra.run.smithi032.stdout:4612 writesame 0xc0d30fb thru 0xc0d3334 data_size 0xa(0x23a bytes) 2024-09-13T09:18:45.091 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:45.091 INFO:teuthology.orchestra.run.smithi032.stdout:4616 write 0x28aa138 thru 0x28b7a7e (0xd947 bytes) 2024-09-13T09:18:45.094 INFO:teuthology.orchestra.run.smithi032.stdout:4617 write 0x8fd2524 thru 0x8fd8c44 (0x6721 bytes) 2024-09-13T09:18:45.098 INFO:teuthology.orchestra.run.smithi032.stdout:4619 write 0x2afa411 thru 0x2afa7f0 (0x3e0 bytes) 2024-09-13T09:18:45.101 INFO:teuthology.orchestra.run.smithi032.stdout:4621 read 0x75d19e5 thru 0x75df472 (0xda8e bytes) 2024-09-13T09:18:45.102 INFO:teuthology.orchestra.run.smithi032.stdout:4622 read 0x9060106 thru 0x9069293 (0x918e bytes) 2024-09-13T09:18:45.103 INFO:teuthology.orchestra.run.smithi032.stdout:4623 writesame 0x2585c7d thru 0x258c2a8 data_size 0x1a(0x662c bytes) 2024-09-13T09:18:45.108 INFO:teuthology.orchestra.run.smithi032.stdout:4624 clone 32 order 18 su 32768 sc 2 2024-09-13T09:18:45.130 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:45.126+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:45.144 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:45.144 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:45.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:45.144 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:45.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.144 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:18:45.144 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:45.144 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 76.456638143999996, 2024-09-13T09:18:45.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.21372158699999999, 2024-09-13T09:18:45.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:45.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10 2024-09-13T09:18:45.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.145 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:45.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685363+0000", 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685372+0000", 2024-09-13T09:18:45.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4590000000000004e-06 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685394+0000", 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1616000000000001e-05 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685407+0000", 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3675e-05 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685484+0000", 2024-09-13T09:18:45.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7065999999999997e-05 2024-09-13T09:18:45.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:18:45.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685493+0000", 2024-09-13T09:18:45.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8070000000000006e-06 2024-09-13T09:18:45.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690770+0000", 2024-09-13T09:18:45.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052763869999999996 2024-09-13T09:18:45.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690940+0000", 2024-09-13T09:18:45.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017001900000000001 2024-09-13T09:18:45.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:45.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.691238+0000", 2024-09-13T09:18:45.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029794499999999998 2024-09-13T09:18:45.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:45.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.692057+0000", 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081943000000000001 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.898933+0000", 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.20687588200000001 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899026+0000", 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2777999999999998e-05 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899086+0000", 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0188e-05 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:14590 3.1d 3:be60f898:::rbd_data.11853ecd5f5f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13672152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.600789704, 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052557382, 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:45.151 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.152 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 14590 2024-09-13T09:18:45.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.152 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:45.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:45.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541207+0000", 2024-09-13T09:18:45.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-13T09:18:45.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562477+0000", 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021270155999999998 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562539+0000", 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1851000000000001e-05 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562565+0000", 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6194e-05 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562702+0000", 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013731299999999999 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.563071+0000", 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036892899999999999 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.565167+0000", 2024-09-13T09:18:45.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020961959999999998 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.585261+0000", 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020093443999999999 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593596+0000", 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083352849999999996 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593696+0000", 2024-09-13T09:18:45.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100021 2024-09-13T09:18:45.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593770+0000", 2024-09-13T09:18:45.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3889999999999999e-05 2024-09-13T09:18:45.157 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.157 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.157 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.157 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:14584 3.2 e39/22 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head v 39'64, mlcod=38'62)", 2024-09-13T09:18:45.157 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.598804057999999, 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051546091000000002, 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543197+0000", 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588691+0000", 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045493475999999998 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588711+0000", 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0485000000000001e-05 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588721+0000", 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.499e-06 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588766+0000", 2024-09-13T09:18:45.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5228000000000001e-05 2024-09-13T09:18:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588788+0000", 2024-09-13T09:18:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2503000000000001e-05 2024-09-13T09:18:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594671+0000", 2024-09-13T09:18:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058822809999999996 2024-09-13T09:18:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594744+0000", 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3776000000000007e-05 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:22580 3.1c 3:39876c9f:::rbd_data.1185d6f8ec87.000000000000000f:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.289551267999997, 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050372896, 2024-09-13T09:18:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22580 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852449+0000", 2024-09-13T09:18:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852462+0000", 2024-09-13T09:18:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3756e-05 2024-09-13T09:18:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852495+0000", 2024-09-13T09:18:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2435999999999999e-05 2024-09-13T09:18:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852516+0000", 2024-09-13T09:18:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1347000000000001e-05 2024-09-13T09:18:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852644+0000", 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012777200000000001 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852784+0000", 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013980500000000001 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.860936+0000", 2024-09-13T09:18:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081526859999999993 2024-09-13T09:18:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902522+0000", 2024-09-13T09:18:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041585203000000001 2024-09-13T09:18:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902560+0000", 2024-09-13T09:18:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8603000000000003e-05 2024-09-13T09:18:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902708+0000", 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014740900000000001 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902824+0000", 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116519 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54746 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8011776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.9106207919999996, 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041426971999999999, 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54746 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231381+0000", 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231384+0000", 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.599e-06 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231390+0000", 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231395+0000", 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0170000000000004e-06 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272716+0000", 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041321670999999997 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272767+0000", 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0447e-05 2024-09-13T09:18:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272809+0000", 2024-09-13T09:18:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1659999999999998e-05 2024-09-13T09:18:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54986 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6348800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8997121929999992, 2024-09-13T09:18:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041375935000000003, 2024-09-13T09:18:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54986 2024-09-13T09:18:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242292+0000", 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3039999999999999e-06 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242298+0000", 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242303+0000", 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8359999999999999e-06 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283539+0000", 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041235330000000001 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283593+0000", 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4926000000000002e-05 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283666+0000", 2024-09-13T09:18:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2718000000000004e-05 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54987 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6447104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8996950110000004, 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041604369000000002, 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54987 2024-09-13T09:18:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242309+0000", 2024-09-13T09:18:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1160000000000002e-06 2024-09-13T09:18:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242317+0000", 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9799999999999998e-06 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242323+0000", 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6439999999999997e-06 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283711+0000", 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388085999999998 2024-09-13T09:18:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283766+0000", 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5112999999999999e-05 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283912+0000", 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014575300000000001 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54988 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6545408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.899675191, 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041865404000000002, 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54988 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242329+0000", 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242335+0000", 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:18:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242341+0000", 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9440000000000002e-06 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283952+0000", 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041610981999999998 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284076+0000", 2024-09-13T09:18:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124356 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284193+0000", 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116345 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54989 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6643712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8996570770000005, 2024-09-13T09:18:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042123657000000002, 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54989 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242348+0000", 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.655e-06 2024-09-13T09:18:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242354+0000", 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2269999999999998e-06 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242360+0000", 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6170000000000003e-06 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284248+0000", 2024-09-13T09:18:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041887830000000001 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284340+0000", 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1358000000000002e-05 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284469+0000", 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012932300000000001 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54990 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6742016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8996237869999995, 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042278333000000001, 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54990 2024-09-13T09:18:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242378+0000", 2024-09-13T09:18:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242381+0000", 2024-09-13T09:18:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0680000000000001e-06 2024-09-13T09:18:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242392+0000", 2024-09-13T09:18:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0762e-05 2024-09-13T09:18:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242398+0000", 2024-09-13T09:18:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1040000000000003e-06 2024-09-13T09:18:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284506+0000", 2024-09-13T09:18:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042107983000000002 2024-09-13T09:18:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284564+0000", 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.821e-05 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284657+0000", 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2588000000000002e-05 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54991 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6840320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8995989959999999, 2024-09-13T09:18:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042489459, 2024-09-13T09:18:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54991 2024-09-13T09:18:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242406+0000", 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242412+0000", 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5919999999999997e-06 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242425+0000", 2024-09-13T09:18:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2713e-05 2024-09-13T09:18:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284694+0000", 2024-09-13T09:18:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042269558999999998 2024-09-13T09:18:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284784+0000", 2024-09-13T09:18:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9999000000000003e-05 2024-09-13T09:18:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284893+0000", 2024-09-13T09:18:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108422 2024-09-13T09:18:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54992 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6938624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8995726249999993, 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042636897999999999, 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54992 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242429+0000", 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242432+0000", 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7599999999999998e-06 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242439+0000", 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5760000000000002e-06 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242448+0000", 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284935+0000", 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042486949000000003 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284991+0000", 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6471000000000002e-05 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285067+0000", 2024-09-13T09:18:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5543000000000005e-05 2024-09-13T09:18:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54993 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7036928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8995497480000001, 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042867959999999997, 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54993 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242452+0000", 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242455+0000", 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4360000000000001e-06 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242463+0000", 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8220000000000007e-06 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242468+0000", 2024-09-13T09:18:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9479999999999999e-06 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285107+0000", 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042639415999999999 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285196+0000", 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8721000000000007e-05 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285321+0000", 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012495 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54994 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7135232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8995225839999996, 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043009622999999997, 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54994 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242482+0000", 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242486+0000", 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1049999999999997e-06 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242489+0000", 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.004e-06 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285354+0000", 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.04286479 2024-09-13T09:18:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285413+0000", 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9778000000000003e-05 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285489+0000", 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6141000000000002e-05 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54995 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7233536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8995112130000003, 2024-09-13T09:18:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043166641999999998, 2024-09-13T09:18:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54995 2024-09-13T09:18:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242492+0000", 2024-09-13T09:18:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.437e-06 2024-09-13T09:18:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242496+0000", 2024-09-13T09:18:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:18:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242498+0000", 2024-09-13T09:18:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5950000000000001e-06 2024-09-13T09:18:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285528+0000", 2024-09-13T09:18:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043029755000000003 2024-09-13T09:18:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285585+0000", 2024-09-13T09:18:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:18:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285658+0000", 2024-09-13T09:18:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2713999999999995e-05 2024-09-13T09:18:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54996 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7331840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8995018980000005, 2024-09-13T09:18:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043333782000000001, 2024-09-13T09:18:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54996 2024-09-13T09:18:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242500+0000", 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242502+0000", 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3599999999999999e-06 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242505+0000", 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1439999999999999e-06 2024-09-13T09:18:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242507+0000", 2024-09-13T09:18:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.605e-06 2024-09-13T09:18:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285698+0000", 2024-09-13T09:18:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043190911999999998 2024-09-13T09:18:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285754+0000", 2024-09-13T09:18:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5541999999999998e-05 2024-09-13T09:18:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285834+0000", 2024-09-13T09:18:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0426000000000006e-05 2024-09-13T09:18:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54997 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8994928089999998, 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043496174999999998, 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54997 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242509+0000", 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242511+0000", 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.313e-06 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242514+0000", 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242516+0000", 2024-09-13T09:18:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3999999999999999e-06 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285875+0000", 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043358578000000002 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285929+0000", 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4602999999999999e-05 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286006+0000", 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6429999999999998e-05 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54998 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7528448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8994001970000003, 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043666469999999999, 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54998 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242518+0000", 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999161 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242606+0000", 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7607000000000003e-05 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242617+0000", 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0882e-05 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242624+0000", 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9009999999999997e-06 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286046+0000", 2024-09-13T09:18:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043422308 2024-09-13T09:18:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286106+0000", 2024-09-13T09:18:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0285000000000003e-05 2024-09-13T09:18:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286269+0000", 2024-09-13T09:18:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000162578 2024-09-13T09:18:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:55006 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8314880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8960115290000008, 2024-09-13T09:18:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041461771000000001, 2024-09-13T09:18:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 55006 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245993+0000", 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2869999999999999e-06 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245999+0000", 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0159999999999997e-06 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.246004+0000", 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287345+0000", 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041341217 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287398+0000", 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2840000000000002e-05 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287453+0000", 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4746000000000001e-05 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:68662 3.1d 3:be8be2fa:::rbd_data.118556027ff4.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:18:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:18:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9849560880000001, 2024-09-13T09:18:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.044923689000000003, 2024-09-13T09:18:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 68662 2024-09-13T09:18:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157045+0000", 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157052+0000", 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8750000000000002e-06 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157066+0000", 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3815000000000001e-05 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157073+0000", 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.64e-06 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157145+0000", 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2517999999999999e-05 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157267+0000", 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012185599999999999 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.164012+0000", 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0067447660000000001 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201805+0000", 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037793528999999999 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201881+0000", 2024-09-13T09:18:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5678000000000003e-05 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201933+0000", 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2346999999999999e-05 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201970+0000", 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6532000000000002e-05 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.246 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:45.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:45.248 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:45.246+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:18:45.348 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:45.346+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:45.428 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:45.428 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:45.444 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:45.449 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:45.446+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:45.467 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone31 from 0xe47e7b7 (overlap 0xc98a48) to 0x14276d 2024-09-13T09:18:45.549 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:45.546+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:45.587 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:45.587 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:45.587 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:45.587 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:45.587 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:45.602 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:45.650 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:45.646+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:45.746 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-13T09:18:45.751 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:45.750+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:45.776 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:45.776 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:45.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:45.776 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:45.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.776 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:50082 3.10 3:0ec43ecb:::rbd_object_map.1185172eb410:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T09:18:45.776 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:45.776 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.411763139, 2024-09-13T09:18:45.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80943783999999996, 2024-09-13T09:18:45.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:45.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 50082 2024-09-13T09:18:45.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.777 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362619+0000", 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362632+0000", 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3338e-05 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362657+0000", 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5207000000000001e-05 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362669+0000", 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1768e-05 2024-09-13T09:18:45.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362738+0000", 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9486999999999994e-05 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362748+0000", 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0010000000000001e-05 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169473+0000", 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80672482300000004 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169884+0000", 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041114299999999999 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.170206+0000", 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032127700000000002 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171827+0000", 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016217880000000001 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171871+0000", 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171967+0000", 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5752000000000001e-05 2024-09-13T09:18:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.172059+0000", 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2082000000000004e-05 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56772 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 4112384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.34407502, 2024-09-13T09:18:45.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054477114, 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56772 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430309+0000", 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430312+0000", 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.255e-06 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430319+0000", 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4760000000000001e-06 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430324+0000", 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9799999999999998e-06 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484634+0000", 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054310031000000002 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484697+0000", 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3355999999999994e-05 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484787+0000", 2024-09-13T09:18:45.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9350000000000003e-05 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56785 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5390336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340291538000001, 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054655007999999998, 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56785 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434094+0000", 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.471e-06 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434097+0000", 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0740000000000001e-06 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434100+0000", 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7829999999999999e-06 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488422+0000", 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054321855000000002 2024-09-13T09:18:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488563+0000", 2024-09-13T09:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014154 2024-09-13T09:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488748+0000", 2024-09-13T09:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018461399999999999 2024-09-13T09:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56786 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340282156000001, 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054950539999999999, 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56786 2024-09-13T09:18:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434104+0000", 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4020000000000001e-06 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434107+0000", 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.072e-06 2024-09-13T09:18:45.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434109+0000", 2024-09-13T09:18:45.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.717e-06 2024-09-13T09:18:45.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488795+0000", 2024-09-13T09:18:45.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054685661000000003 2024-09-13T09:18:45.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488869+0000", 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4265999999999997e-05 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489053+0000", 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018374299999999999 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56787 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5586944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340271896999999, 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055115725999999997, 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56787 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434112+0000", 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434114+0000", 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.308e-06 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434117+0000", 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434119+0000", 2024-09-13T09:18:45.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.588e-06 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489097+0000", 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054977824000000002 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489146+0000", 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8503e-05 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489228+0000", 2024-09-13T09:18:45.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2694000000000005e-05 2024-09-13T09:18:45.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.802 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.802 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56788 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.802 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:45.802 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340263006000001, 2024-09-13T09:18:45.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055437562000000003, 2024-09-13T09:18:45.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56788 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434121+0000", 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434123+0000", 2024-09-13T09:18:45.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2190000000000001e-06 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434126+0000", 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434128+0000", 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8250000000000001e-06 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489293+0000", 2024-09-13T09:18:45.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055164092999999997 2024-09-13T09:18:45.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489428+0000", 2024-09-13T09:18:45.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013572599999999999 2024-09-13T09:18:45.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489559+0000", 2024-09-13T09:18:45.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013086 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56789 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340253857, 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055674077000000002, 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56789 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434132+0000", 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.302e-06 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434136+0000", 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434139+0000", 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489583+0000", 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055443994000000003 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489699+0000", 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116611 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489805+0000", 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105331 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.811 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56790 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340243482, 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055716839999999997, 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56790 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434142+0000", 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2559999999999999e-06 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434145+0000", 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.96e-06 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434148+0000", 2024-09-13T09:18:45.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6479999999999999e-06 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489851+0000", 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055702792000000001 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489858+0000", 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3900000000000004e-06 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56791 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340234710000001, 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055740029000000003, 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56791 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.816 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434151+0000", 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434154+0000", 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0139999999999999e-06 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434157+0000", 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6699999999999998e-06 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489884+0000", 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055727511 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489890+0000", 2024-09-13T09:18:45.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7200000000000003e-06 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56792 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340225781000001, 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055760363, 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56792 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:45.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434163+0000", 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434167+0000", 2024-09-13T09:18:45.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.292e-06 2024-09-13T09:18:45.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434169+0000", 2024-09-13T09:18:45.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:18:45.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489914+0000", 2024-09-13T09:18:45.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055744415999999998 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489919+0000", 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56793 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:45.824 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340212261, 2024-09-13T09:18:45.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055774049999999999, 2024-09-13T09:18:45.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56793 2024-09-13T09:18:45.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.825 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434173+0000", 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-06 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434177+0000", 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1070000000000001e-06 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434179+0000", 2024-09-13T09:18:45.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.666e-06 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489941+0000", 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055762071000000003 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489946+0000", 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56794 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340203152999999, 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055791704999999997, 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56794 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.829 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434182+0000", 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1629999999999999e-06 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434185+0000", 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9950000000000001e-06 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434188+0000", 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489968+0000", 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055779789000000003 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489973+0000", 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2800000000000003e-06 2024-09-13T09:18:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.833 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.833 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56795 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.833 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:45.833 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340194217000001, 2024-09-13T09:18:45.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055809612000000001, 2024-09-13T09:18:45.833 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.833 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.833 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.833 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56795 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:45.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434191+0000", 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.212e-06 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434194+0000", 2024-09-13T09:18:45.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.078e-06 2024-09-13T09:18:45.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434198+0000", 2024-09-13T09:18:45.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9820000000000002e-06 2024-09-13T09:18:45.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489995+0000", 2024-09-13T09:18:45.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055796363000000002 2024-09-13T09:18:45.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490000+0000", 2024-09-13T09:18:45.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1839999999999998e-06 2024-09-13T09:18:45.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.837 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.837 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56796 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.837 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:45.838 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340183956000001, 2024-09-13T09:18:45.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055825859999999998, 2024-09-13T09:18:45.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56796 2024-09-13T09:18:45.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.838 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434200+0000", 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434202+0000", 2024-09-13T09:18:45.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.204e-06 2024-09-13T09:18:45.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434205+0000", 2024-09-13T09:18:45.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9689999999999999e-06 2024-09-13T09:18:45.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434207+0000", 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490021+0000", 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055813929999999998 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490027+0000", 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.259e-06 2024-09-13T09:18:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56797 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.340175237, 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055843649000000002, 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56797 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:45.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434210+0000", 2024-09-13T09:18:45.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2389999999999999e-06 2024-09-13T09:18:45.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434213+0000", 2024-09-13T09:18:45.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:18:45.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434216+0000", 2024-09-13T09:18:45.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7800000000000001e-06 2024-09-13T09:18:45.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490048+0000", 2024-09-13T09:18:45.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055831981000000003 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490053+0000", 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8640000000000004e-06 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56807 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.338495078999999, 2024-09-13T09:18:45.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054407720999999999, 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56807 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435889+0000", 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435891+0000", 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.217e-06 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435894+0000", 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.067e-06 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435896+0000", 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490292+0000", 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054395840000000001 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490297+0000", 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56808 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.338486252999999, 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05442524, 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56808 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:45.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:45.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435900+0000", 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435903+0000", 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435905+0000", 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490319+0000", 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054413396000000003 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490324+0000", 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:18:45.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56809 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.338476197, 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054442786, 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56809 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435910+0000", 2024-09-13T09:18:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:18:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435913+0000", 2024-09-13T09:18:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1470000000000001e-06 2024-09-13T09:18:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435915+0000", 2024-09-13T09:18:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6769999999999999e-06 2024-09-13T09:18:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490346+0000", 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054430574000000002 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490351+0000", 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2700000000000004e-06 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56810 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.338466898, 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054459855000000001, 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56810 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435917+0000", 2024-09-13T09:18:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435919+0000", 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.457e-06 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435924+0000", 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435929+0000", 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8300000000000003e-06 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490373+0000", 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054443841999999999 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490378+0000", 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8509999999999998e-06 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56811 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.338437796999999, 2024-09-13T09:18:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054456683999999998, 2024-09-13T09:18:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56811 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435946+0000", 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435949+0000", 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435956+0000", 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8719999999999996e-06 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435962+0000", 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000004e-06 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490398+0000", 2024-09-13T09:18:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054436824000000002 2024-09-13T09:18:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490403+0000", 2024-09-13T09:18:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0710000000000002e-06 2024-09-13T09:18:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:45.868 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:45.869 INFO:teuthology.orchestra.run.smithi032.stdout:4625 trunc from 0xe47e7b7 to 0x1dbfd21 2024-09-13T09:18:45.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:45.871 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:45.866+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:45.917 INFO:teuthology.orchestra.run.smithi032.stdout:4629 read 0x471eb thru 0x53518 (0xc32e bytes) 2024-09-13T09:18:45.919 INFO:teuthology.orchestra.run.smithi032.stdout:4631 write 0x1a5e618 thru 0x1a6a2f9 (0xbce2 bytes) 2024-09-13T09:18:45.933 INFO:teuthology.orchestra.run.smithi032.stdout:4632 read 0x15d6c91 thru 0x15e179e (0xab0e bytes) 2024-09-13T09:18:45.934 INFO:teuthology.orchestra.run.smithi032.stdout:4635 read 0xf640fa thru 0xf6ed46 (0xac4d bytes) 2024-09-13T09:18:45.966 INFO:teuthology.orchestra.run.smithi032.stdout:4636 trunc from 0x1dbfd21 to 0xce8ba66 2024-09-13T09:18:45.970 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:45.966+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:45.978 INFO:teuthology.orchestra.run.smithi032.stdout:4637 read 0x8c0d1a5 thru 0x8c1490a (0x7766 bytes) 2024-09-13T09:18:45.978 INFO:teuthology.orchestra.run.smithi032.stdout:4639 read 0x72406ef thru 0x724f2f8 (0xec0a bytes) 2024-09-13T09:18:45.978 INFO:teuthology.orchestra.run.smithi032.stdout:4643 read 0x7b9a35e thru 0x7b9dfac (0x3c4f bytes) 2024-09-13T09:18:45.978 INFO:teuthology.orchestra.run.smithi032.stdout:4644 read 0x16d550b thru 0x16d62ef (0xde5 bytes) 2024-09-13T09:18:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:4645 write 0x7967ed1 thru 0x796c311 (0x4441 bytes) 2024-09-13T09:18:45.988 INFO:teuthology.orchestra.run.smithi032.stdout:4646 read 0x43b6f8f thru 0x43bb335 (0x43a7 bytes) 2024-09-13T09:18:45.988 INFO:teuthology.orchestra.run.smithi032.stdout:4647 read 0x48a82d thru 0x497fbf (0xd793 bytes) 2024-09-13T09:18:46.021 INFO:teuthology.orchestra.run.smithi032.stdout:4648 trunc from 0xce8ba66 to 0x11ea0a3 2024-09-13T09:18:46.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:18:46.054 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:46.054 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:46.054 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:46.054 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:46.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:46.079 INFO:teuthology.orchestra.run.smithi032.stdout:4649 writesame 0x4828bc3 thru 0x4836711 data_size 0x1(0xdb4f bytes) 2024-09-13T09:18:46.079 INFO:teuthology.orchestra.run.smithi032.stdout:4650 read 0x44f19f5 thru 0x44f92d3 (0x78df bytes) 2024-09-13T09:18:46.102 INFO:teuthology.orchestra.run.smithi032.stdout:4652 writesame 0x8ea73b6 thru 0x8eb4bc9 data_size 0x2(0xd814 bytes) 2024-09-13T09:18:46.102 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:46.102 INFO:teuthology.orchestra.run.smithi032.stdout:4654 punch from 0x50dae2e to 0x50de168, (0x333a bytes) 2024-09-13T09:18:46.102 INFO:teuthology.orchestra.run.smithi032.stdout:4658 read 0x41c9bfc thru 0x41d1594 (0x7999 bytes) 2024-09-13T09:18:46.102 INFO:teuthology.orchestra.run.smithi032.stdout:4659 punch from 0x8d4f40b to 0x8d57a38, (0x862d bytes) 2024-09-13T09:18:46.103 INFO:teuthology.orchestra.run.smithi032.stdout:4660 read 0x4475540 thru 0x4477452 (0x1f13 bytes) 2024-09-13T09:18:46.110 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:46.106+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:46.129 INFO:teuthology.orchestra.run.smithi032.stdout:4661 write 0xebec7ca thru 0xebf7526 (0xad5d bytes) 2024-09-13T09:18:46.139 INFO:teuthology.orchestra.run.smithi032.stdout:4662 write 0x6fcf811 thru 0x6fd1a72 (0x2262 bytes) 2024-09-13T09:18:46.146 INFO:teuthology.orchestra.run.smithi032.stdout:4664 write 0xba1c50 thru 0xbac769 (0xab1a bytes) 2024-09-13T09:18:46.158 INFO:teuthology.orchestra.run.smithi032.stdout:4665 read 0xda0254 thru 0xdaa3ac (0xa159 bytes) 2024-09-13T09:18:46.160 INFO:teuthology.orchestra.run.smithi032.stdout:4667 writesame 0x2e7efb9 thru 0x2e8bddf data_size 0x5(0xce27 bytes) 2024-09-13T09:18:46.172 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:46.170+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:46.174 INFO:teuthology.orchestra.run.smithi032.stdout:4668 write 0xb99c8d7 thru 0xb99ede9 (0x2513 bytes) 2024-09-13T09:18:46.180 INFO:teuthology.orchestra.run.smithi032.stdout:4669 write 0x294d602 thru 0x295351c (0x5f1b bytes) 2024-09-13T09:18:46.186 INFO:teuthology.orchestra.run.smithi032.stdout:4670 write 0x4b85149 thru 0x4b92835 (0xd6ed bytes) 2024-09-13T09:18:46.192 INFO:teuthology.orchestra.run.smithi032.stdout:4671 punch from 0x8cda995 to 0x8ce8a5d, (0xe0c8 bytes) 2024-09-13T09:18:46.192 INFO:teuthology.orchestra.run.smithi032.stdout:4672 punch from 0x5ff2166 to 0x5fff5f3, (0xd48d bytes) 2024-09-13T09:18:46.192 INFO:teuthology.orchestra.run.smithi032.stdout:4674 write 0x8f65f63 thru 0x8f66e3c (0xeda bytes) 2024-09-13T09:18:46.196 INFO:teuthology.orchestra.run.smithi032.stdout:4675 read 0x1b965d7 thru 0x1ba0730 (0xa15a bytes) 2024-09-13T09:18:46.197 INFO:teuthology.orchestra.run.smithi032.stdout:4676 writesame 0x7919aed thru 0x7925a60 data_size 0x2(0xbf74 bytes) 2024-09-13T09:18:46.210 INFO:teuthology.orchestra.run.smithi032.stdout:4677 read 0x510e585 thru 0x51176a9 (0x9125 bytes) 2024-09-13T09:18:46.211 INFO:teuthology.orchestra.run.smithi032.stdout:4678 punch from 0x446114d to 0x446ca62, (0xb915 bytes) 2024-09-13T09:18:46.211 INFO:teuthology.orchestra.run.smithi032.stdout:4679 write 0xb54d5b3 thru 0xb554887 (0x72d5 bytes) 2024-09-13T09:18:46.214 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:46.214 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:46.217 INFO:teuthology.orchestra.run.smithi032.stdout:4680 read 0x1b4b714 thru 0x1b59b11 (0xe3fe bytes) 2024-09-13T09:18:46.217 INFO:teuthology.orchestra.run.smithi032.stdout:4682 read 0x884d6f6 thru 0x885a56d (0xce78 bytes) 2024-09-13T09:18:46.217 INFO:teuthology.orchestra.run.smithi032.stdout:4683 read 0x9268979 thru 0x9277984 (0xf00c bytes) 2024-09-13T09:18:46.218 INFO:teuthology.orchestra.run.smithi032.stdout:4687 read 0x3d1f694 thru 0x3d2cad7 (0xd444 bytes) 2024-09-13T09:18:46.218 INFO:teuthology.orchestra.run.smithi032.stdout:4689 write 0x8ea0634 thru 0x8eabbe7 (0xb5b4 bytes) 2024-09-13T09:18:46.224 INFO:teuthology.orchestra.run.smithi032.stdout:4692 punch from 0x756aa51 to 0x756c112, (0x16c1 bytes) 2024-09-13T09:18:46.224 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:46.224 INFO:teuthology.orchestra.run.smithi032.stdout:4694 punch from 0x5ae5c4b to 0x5af29e3, (0xcd98 bytes) 2024-09-13T09:18:46.224 INFO:teuthology.orchestra.run.smithi032.stdout:4695 punch from 0xef9847 to 0xf06cf9, (0xd4b2 bytes) 2024-09-13T09:18:46.224 INFO:teuthology.orchestra.run.smithi032.stdout:4696 punch from 0xe1f71f2 to 0xe1feac5, (0x78d3 bytes) 2024-09-13T09:18:46.224 INFO:teuthology.orchestra.run.smithi032.stdout:4697 read 0xb454e7 thru 0xb52a6a (0xd584 bytes) 2024-09-13T09:18:46.225 INFO:teuthology.orchestra.run.smithi032.stdout:4700 write 0x25f45d9 thru 0x25fd329 (0x8d51 bytes) 2024-09-13T09:18:46.231 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:46.231 INFO:teuthology.orchestra.run.smithi032.stdout:4703 write 0x8c0002b thru 0x8c0ee4d (0xee23 bytes) 2024-09-13T09:18:46.232 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:46.232 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:46.237 INFO:teuthology.orchestra.run.smithi032.stdout:4705 punch from 0x455f144 to 0x4567e95, (0x8d51 bytes) 2024-09-13T09:18:46.238 INFO:teuthology.orchestra.run.smithi032.stdout:4706 write 0x612f717 thru 0x613a7a1 (0xb08b bytes) 2024-09-13T09:18:46.252 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:46.254 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:46.254 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:46.261 INFO:teuthology.orchestra.run.smithi032.stdout:4707 trunc from 0xebf7527 to 0x3d6da40 2024-09-13T09:18:46.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:46.273 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:46.270+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:46.273 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:46.291 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:46.296 INFO:teuthology.orchestra.run.smithi032.stdout:4709 write 0x9088955 thru 0x909032a (0x79d6 bytes) 2024-09-13T09:18:46.300 INFO:teuthology.orchestra.run.smithi032.stdout:4710 write 0xcf8677 thru 0xcffaba (0x7444 bytes) 2024-09-13T09:18:46.306 INFO:teuthology.orchestra.run.smithi032.stdout:4712 punch from 0x19094c6 to 0x190cdaf, (0x38e9 bytes) 2024-09-13T09:18:46.306 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:46.306 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:46.308 INFO:teuthology.orchestra.run.smithi032.stdout:4715 trunc from 0x909032b to 0x741bafb 2024-09-13T09:18:46.310 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:46.329 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:46.329 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:46.330 INFO:teuthology.orchestra.run.smithi032.stdout:4716 trunc from 0x741bafb to 0x49f423 2024-09-13T09:18:46.351 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:46.371 INFO:teuthology.orchestra.run.smithi032.stdout:4717 writesame 0x6b6a4db thru 0x6b6a4f6 data_size 0x4(0x1c bytes) 2024-09-13T09:18:46.373 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:46.370+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:46.389 INFO:teuthology.orchestra.run.smithi032.stdout:4718 write 0xc4f45dd thru 0xc4f638f (0x1db3 bytes) 2024-09-13T09:18:46.395 INFO:teuthology.orchestra.run.smithi032.stdout:4719 trunc from 0xc4f6390 to 0xb47b88a 2024-09-13T09:18:46.405 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:46.405 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:46.410 INFO:teuthology.orchestra.run.smithi032.stdout:4720 read 0x22b68cd thru 0x22b9548 (0x2c7c bytes) 2024-09-13T09:18:46.411 INFO:teuthology.orchestra.run.smithi032.stdout:4722 read 0x933308d thru 0x933c692 (0x9606 bytes) 2024-09-13T09:18:46.411 INFO:teuthology.orchestra.run.smithi032.stdout:4723 read 0x2e07d7d thru 0x2e175e1 (0xf865 bytes) 2024-09-13T09:18:46.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:46.423 INFO:teuthology.orchestra.run.smithi032.stdout:4724 trunc from 0xb47b88a to 0x6263dd1 2024-09-13T09:18:46.436 INFO:teuthology.orchestra.run.smithi032.stdout:4725 trunc from 0x6263dd1 to 0x90f5b8b 2024-09-13T09:18:46.444 INFO:teuthology.orchestra.run.smithi032.stdout:4730 read 0x7322fe0 thru 0x732475e (0x177f bytes) 2024-09-13T09:18:46.452 INFO:teuthology.orchestra.run.smithi032.stdout:4731 trunc from 0x90f5b8b to 0x40431b8 2024-09-13T09:18:46.461 INFO:teuthology.orchestra.run.smithi032.stdout:4732 read 0x3acaf5b thru 0x3ad06ca (0x5770 bytes) 2024-09-13T09:18:46.461 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:46.461 INFO:teuthology.orchestra.run.smithi032.stdout:4736 read 0x1a170e7 thru 0x1a17dde (0xcf8 bytes) 2024-09-13T09:18:46.461 INFO:teuthology.orchestra.run.smithi032.stdout:4737 read 0x2a6e3af thru 0x2a6ea09 (0x65b bytes) 2024-09-13T09:18:46.468 INFO:teuthology.orchestra.run.smithi032.stdout:4738 trunc from 0x40431b8 to 0x633998 2024-09-13T09:18:46.474 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:46.470+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:46.481 INFO:teuthology.orchestra.run.smithi032.stdout:4739 trunc from 0x633998 to 0x317f88f 2024-09-13T09:18:46.490 INFO:teuthology.orchestra.run.smithi032.stdout:4740 punch from 0x3d973d to 0x3e0f23, (0x77e6 bytes) 2024-09-13T09:18:46.490 INFO:teuthology.orchestra.run.smithi032.stdout:4743 write 0x20a35e1 thru 0x20ac58c (0x8fac bytes) 2024-09-13T09:18:46.515 INFO:teuthology.orchestra.run.smithi032.stdout:4744 trunc from 0x317f88f to 0xdb78baf 2024-09-13T09:18:46.531 INFO:teuthology.orchestra.run.smithi032.stdout:4745 write 0x5c15040 thru 0x5c176fd (0x26be bytes) 2024-09-13T09:18:46.539 INFO:teuthology.orchestra.run.smithi032.stdout:4746 punch from 0x4e725ee to 0x4e77810, (0x5222 bytes) 2024-09-13T09:18:46.539 INFO:teuthology.orchestra.run.smithi032.stdout:4748 read 0xaea389c thru 0xaeab176 (0x78db bytes) 2024-09-13T09:18:46.539 INFO:teuthology.orchestra.run.smithi032.stdout:4749 write 0x4c36cb7 thru 0x4c38d25 (0x206f bytes) 2024-09-13T09:18:46.547 INFO:teuthology.orchestra.run.smithi032.stdout:4750 write 0x5a6dfaf thru 0x5a71588 (0x35da bytes) 2024-09-13T09:18:46.562 INFO:teuthology.orchestra.run.smithi032.stdout:4751 trunc from 0xdb78baf to 0x979b023 2024-09-13T09:18:46.574 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:46.570+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:46.579 INFO:teuthology.orchestra.run.smithi032.stdout:4752 trunc from 0x979b023 to 0xbd19473 2024-09-13T09:18:46.584 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:46.584 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:46.592 INFO:teuthology.orchestra.run.smithi032.stdout:4753 write 0x3caad12 thru 0x3cb8f55 (0xe244 bytes) 2024-09-13T09:18:46.600 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:46.603 INFO:teuthology.orchestra.run.smithi032.stdout:4754 read 0x6dabc2a thru 0x6db9389 (0xd760 bytes) 2024-09-13T09:18:46.605 INFO:teuthology.orchestra.run.smithi032.stdout:4756 trunc from 0xbd19473 to 0xb4d678f 2024-09-13T09:18:46.621 INFO:teuthology.orchestra.run.smithi032.stdout:4759 read 0xac9df47 thru 0xaca5de5 (0x7e9f bytes) 2024-09-13T09:18:46.632 INFO:teuthology.orchestra.run.smithi032.stdout:4760 trunc from 0xb4d678f to 0x4b06964 2024-09-13T09:18:46.657 INFO:teuthology.orchestra.run.smithi032.stdout:4761 write 0x46612bc thru 0x46643ad (0x30f2 bytes) 2024-09-13T09:18:46.664 INFO:teuthology.orchestra.run.smithi032.stdout:4763 writesame 0xc2a033 thru 0xc383ab data_size 0x1(0xe379 bytes) 2024-09-13T09:18:46.675 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:46.674+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:46.707 INFO:teuthology.orchestra.run.smithi032.stdout:4765 read 0x20e21 thru 0x2bb34 (0xad14 bytes) 2024-09-13T09:18:46.708 INFO:teuthology.orchestra.run.smithi032.stdout:4766 write 0x3a5f220 thru 0x3a62f19 (0x3cfa bytes) 2024-09-13T09:18:46.718 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:46.718 INFO:teuthology.orchestra.run.smithi032.stdout:4768 punch from 0x411d678 to 0x41209fb, (0x3383 bytes) 2024-09-13T09:18:46.718 INFO:teuthology.orchestra.run.smithi032.stdout:4770 write 0x271e86 thru 0x272adc (0xc57 bytes) 2024-09-13T09:18:46.731 INFO:teuthology.orchestra.run.smithi032.stdout:4771 read 0x1e09288 thru 0x1e187f3 (0xf56c bytes) 2024-09-13T09:18:46.732 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:46.732 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:46.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:46.764 INFO:teuthology.orchestra.run.smithi032.stdout:4775 writesame 0xbe5416e thru 0xbe54882 data_size 0x7(0x715 bytes) 2024-09-13T09:18:46.764 INFO:teuthology.orchestra.run.smithi032.stdout:4776 read 0xb8399eb thru 0xb8485ac (0xebc2 bytes) 2024-09-13T09:18:46.764 INFO:teuthology.orchestra.run.smithi032.stdout:4779 read 0x8c21251 thru 0x8c216de (0x48e bytes) 2024-09-13T09:18:46.765 INFO:teuthology.orchestra.run.smithi032.stdout:4780 punch from 0x2e63a97 to 0x2e643fd, (0x966 bytes) 2024-09-13T09:18:46.765 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:46.776 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:46.770+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:46.778 INFO:teuthology.orchestra.run.smithi032.stdout:4784 trunc from 0xbe54883 to 0x598772a 2024-09-13T09:18:46.793 INFO:teuthology.orchestra.run.smithi032.stdout:4785 punch from 0x2347f07 to 0x234d8c2, (0x59bb bytes) 2024-09-13T09:18:46.811 INFO:teuthology.orchestra.run.smithi032.stdout:4788 trunc from 0x598772a to 0xd6a0a37 2024-09-13T09:18:46.827 INFO:teuthology.orchestra.run.smithi032.stdout:4789 write 0x1774050 thru 0x178296c (0xe91d bytes) 2024-09-13T09:18:46.838 INFO:teuthology.orchestra.run.smithi032.stdout:4790 read 0x3e06065 thru 0x3e06761 (0x6fd bytes) 2024-09-13T09:18:46.838 INFO:teuthology.orchestra.run.smithi032.stdout:4791 punch from 0xb19267d to 0xb19b114, (0x8a97 bytes) 2024-09-13T09:18:46.838 INFO:teuthology.orchestra.run.smithi032.stdout:4792 write 0x2912a0d thru 0x2913936 (0xf2a bytes) 2024-09-13T09:18:46.867 INFO:teuthology.orchestra.run.smithi032.stdout:4793 trunc from 0xd6a0a37 to 0x37ab300 2024-09-13T09:18:46.877 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:46.874+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:46.892 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:46.892 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:46.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:46.922 INFO:teuthology.orchestra.run.smithi032.stdout:4794 write 0x9a8f224 thru 0x9a92d93 (0x3b70 bytes) 2024-09-13T09:18:46.931 INFO:teuthology.orchestra.run.smithi032.stdout:4795 writesame 0x2b03fc2 thru 0x2b05dc4 data_size 0xd(0x1e03 bytes) 2024-09-13T09:18:46.938 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:46.938 INFO:teuthology.orchestra.run.smithi032.stdout:4797 write 0x6d9278f thru 0x6d97710 (0x4f82 bytes) 2024-09-13T09:18:46.944 INFO:teuthology.orchestra.run.smithi032.stdout:4798 read 0x8639c34 thru 0x863d3b2 (0x377f bytes) 2024-09-13T09:18:46.944 INFO:teuthology.orchestra.run.smithi032.stdout:4800 read 0x73a0e08 thru 0x73a294b (0x1b44 bytes) 2024-09-13T09:18:46.944 INFO:teuthology.orchestra.run.smithi032.stdout:4801 punch from 0x5d974ca to 0x5d9fa0d, (0x8543 bytes) 2024-09-13T09:18:46.944 INFO:teuthology.orchestra.run.smithi032.stdout:4802 read 0x823ba89 thru 0x8245616 (0x9b8e bytes) 2024-09-13T09:18:46.944 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:46.944 INFO:teuthology.orchestra.run.smithi032.stdout:4805 read 0x1b2662c thru 0x1b2793c (0x1311 bytes) 2024-09-13T09:18:46.945 INFO:teuthology.orchestra.run.smithi032.stdout:4806 read 0x656260e thru 0x656bf70 (0x9963 bytes) 2024-09-13T09:18:46.945 INFO:teuthology.orchestra.run.smithi032.stdout:4807 punch from 0x184bae3 to 0x18521cd, (0x66ea bytes) 2024-09-13T09:18:46.945 INFO:teuthology.orchestra.run.smithi032.stdout:4810 write 0x89db997 thru 0x89ea772 (0xeddc bytes) 2024-09-13T09:18:46.954 INFO:teuthology.orchestra.run.smithi032.stdout:4811 read 0x69c119 thru 0x69e1a8 (0x2090 bytes) 2024-09-13T09:18:46.954 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:46.960 INFO:teuthology.orchestra.run.smithi032.stdout:4813 trunc from 0x9a92d94 to 0x70edd17 2024-09-13T09:18:46.978 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:46.974+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:46.981 INFO:teuthology.orchestra.run.smithi032.stdout:4814 write 0x1ec29f thru 0x1f9a95 (0xd7f7 bytes) 2024-09-13T09:18:47.014 INFO:teuthology.orchestra.run.smithi032.stdout:4815 write 0xc15af64 thru 0xc1696b7 (0xe754 bytes) 2024-09-13T09:18:47.023 INFO:teuthology.orchestra.run.smithi032.stdout:4816 writesame 0x491c0ac thru 0x492399d data_size 0x2(0x78f2 bytes) 2024-09-13T09:18:47.043 INFO:teuthology.orchestra.run.smithi032.stdout:4817 punch from 0x28df359 to 0x28e3dde, (0x4a85 bytes) 2024-09-13T09:18:47.043 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:47.043 INFO:teuthology.orchestra.run.smithi032.stdout:4819 read 0x930ff10 thru 0x931f70c (0xf7fd bytes) 2024-09-13T09:18:47.043 INFO:teuthology.orchestra.run.smithi032.stdout:4821 punch from 0x750c757 to 0x7516c10, (0xa4b9 bytes) 2024-09-13T09:18:47.043 INFO:teuthology.orchestra.run.smithi032.stdout:4822 write 0x4540f75 thru 0x4547d0f (0x6d9b bytes) 2024-09-13T09:18:47.050 INFO:teuthology.orchestra.run.smithi032.stdout:4825 punch from 0x194f763 to 0x195dc8a, (0xe527 bytes) 2024-09-13T09:18:47.050 INFO:teuthology.orchestra.run.smithi032.stdout:4826 read 0x175934b thru 0x1759c92 (0x948 bytes) 2024-09-13T09:18:47.050 INFO:teuthology.orchestra.run.smithi032.stdout:4828 punch from 0x8eb29fa to 0x8ebdfd3, (0xb5d9 bytes) 2024-09-13T09:18:47.051 INFO:teuthology.orchestra.run.smithi032.stdout:4830 flatten 2024-09-13T09:18:47.061 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:47.061 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:47.079 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:47.074+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:47.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:47.111 INFO:teuthology.orchestra.run.smithi032.stdout:4831 read 0x9be197b thru 0x9be84b4 (0x6b3a bytes) 2024-09-13T09:18:47.126 INFO:teuthology.orchestra.run.smithi032.stdout:4832 writesame 0xd6fe98b thru 0xd707208 data_size 0x1(0x887e bytes) 2024-09-13T09:18:47.126 INFO:teuthology.orchestra.run.smithi032.stdout:4833 writesame 0x81abdc thru 0x827697 data_size 0x4b(0xcabc bytes) 2024-09-13T09:18:47.152 INFO:teuthology.orchestra.run.smithi032.stdout:4834 writesame 0xe9554ea thru 0xe96179d data_size 0x22(0xc2b4 bytes) 2024-09-13T09:18:47.152 INFO:teuthology.orchestra.run.smithi032.stdout:4835 write 0x99d3db0 thru 0x99d56c3 (0x1914 bytes) 2024-09-13T09:18:47.159 INFO:teuthology.orchestra.run.smithi032.stdout:4838 writesame 0xacaf498 thru 0xacb0bea data_size 0x1(0x1753 bytes) 2024-09-13T09:18:47.168 INFO:teuthology.orchestra.run.smithi032.stdout:4840 read 0x9f8206e thru 0x9f86490 (0x4423 bytes) 2024-09-13T09:18:47.168 INFO:teuthology.orchestra.run.smithi032.stdout:4842 read 0x8388925 thru 0x838e028 (0x5704 bytes) 2024-09-13T09:18:47.168 INFO:teuthology.orchestra.run.smithi032.stdout:4845 writesame 0xbab41b2 thru 0xbabda77 data_size 0x5(0x98c6 bytes) 2024-09-13T09:18:47.179 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:47.174+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:47.180 INFO:teuthology.orchestra.run.smithi032.stdout:4847 trunc from 0xe96179e to 0xcbd94ec 2024-09-13T09:18:47.191 INFO:teuthology.orchestra.run.smithi032.stdout:4849 write 0x145607d thru 0x145920b (0x318f bytes) 2024-09-13T09:18:47.199 INFO:teuthology.orchestra.run.smithi032.stdout:4851 write 0xc0c7fd5 thru 0xc0cdd5a (0x5d86 bytes) 2024-09-13T09:18:47.207 INFO:teuthology.orchestra.run.smithi032.stdout:4852 read 0x28287d2 thru 0x28370cb (0xe8fa bytes) 2024-09-13T09:18:47.207 INFO:teuthology.orchestra.run.smithi032.stdout:4853 read 0x6ab606b thru 0x6ac0e87 (0xae1d bytes) 2024-09-13T09:18:47.207 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:47.208 INFO:teuthology.orchestra.run.smithi032.stdout:4855 punch from 0x3f8318c to 0x3f8b50c, (0x8380 bytes) 2024-09-13T09:18:47.208 INFO:teuthology.orchestra.run.smithi032.stdout:4856 punch from 0xb4d5596 to 0xb4da0de, (0x4b48 bytes) 2024-09-13T09:18:47.208 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:47.208 INFO:teuthology.orchestra.run.smithi032.stdout:4858 write 0x5360aa0 thru 0x536527f (0x47e0 bytes) 2024-09-13T09:18:47.214 INFO:teuthology.orchestra.run.smithi032.stdout:4863 punch from 0xbc0865c to 0xbc11465, (0x8e09 bytes) 2024-09-13T09:18:47.215 INFO:teuthology.orchestra.run.smithi032.stdout:4864 write 0x6015af4 thru 0x6016dc2 (0x12cf bytes) 2024-09-13T09:18:47.229 INFO:teuthology.orchestra.run.smithi032.stdout:4865 trunc from 0xcbd94ec to 0x9883ea9 2024-09-13T09:18:47.248 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:47.249 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:47.254 INFO:teuthology.orchestra.run.smithi032.stdout:4866 write 0x1765af6 thru 0x177007a (0xa585 bytes) 2024-09-13T09:18:47.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:47.266 INFO:teuthology.orchestra.run.smithi032.stdout:4868 trunc from 0x9883ea9 to 0x64124b6 2024-09-13T09:18:47.280 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:47.274+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:47.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:18:47.287 INFO:teuthology.orchestra.run.smithi032.stdout:4869 write 0x320aacd thru 0x3210450 (0x5984 bytes) 2024-09-13T09:18:47.296 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:47.296 INFO:teuthology.orchestra.run.smithi032.stdout:4871 writesame 0x47b3523 thru 0x47b9c54 data_size 0x15(0x6732 bytes) 2024-09-13T09:18:47.329 INFO:teuthology.orchestra.run.smithi032.stdout:4872 write 0xc779e89 thru 0xc7844ed (0xa665 bytes) 2024-09-13T09:18:47.337 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:47.353 INFO:teuthology.orchestra.run.smithi032.stdout:4874 write 0xe0a533c thru 0xe0aad3b (0x5a00 bytes) 2024-09-13T09:18:47.362 INFO:teuthology.orchestra.run.smithi032.stdout:4875 write 0xdbcea6b thru 0xdbdbec9 (0xd45f bytes) 2024-09-13T09:18:47.372 INFO:teuthology.orchestra.run.smithi032.stdout:4877 read 0x45c1745 thru 0x45d0138 (0xe9f4 bytes) 2024-09-13T09:18:47.372 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:47.372 INFO:teuthology.orchestra.run.smithi032.stdout:4881 write 0xa6c080d thru 0xa6c66c4 (0x5eb8 bytes) 2024-09-13T09:18:47.378 INFO:teuthology.orchestra.run.smithi032.stdout:4882 writesame 0x4968f19 thru 0x496f89f data_size 0x5(0x6987 bytes) 2024-09-13T09:18:47.380 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:47.378+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:47.403 INFO:teuthology.orchestra.run.smithi032.stdout:4884 trunc from 0xe0aad3c to 0x414355e 2024-09-13T09:18:47.416 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:47.416 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:47.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:47.445 INFO:teuthology.orchestra.run.smithi032.stdout:4885 write 0x6219b36 thru 0x62200da (0x65a5 bytes) 2024-09-13T09:18:47.466 INFO:teuthology.orchestra.run.smithi032.stdout:4886 writesame 0x860b8a5 thru 0x8614f62 data_size 0x55(0x96be bytes) 2024-09-13T09:18:47.466 INFO:teuthology.orchestra.run.smithi032.stdout:4888 read 0x207ed07 thru 0x207f5b9 (0x8b3 bytes) 2024-09-13T09:18:47.466 INFO:teuthology.orchestra.run.smithi032.stdout:4889 read 0x7ad00e1 thru 0x7ad24cf (0x23ef bytes) 2024-09-13T09:18:47.466 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:47.466 INFO:teuthology.orchestra.run.smithi032.stdout:4891 clone 33 order 21 su 16384 sc 2 2024-09-13T09:18:47.482 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:47.478+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:47.566 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:47.566 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:47.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:47.623 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:47.622+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:47.684 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:47.678+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:47.752 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:47.752 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:47.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:47.785 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:47.778+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:47.833 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-13T09:18:47.841 INFO:teuthology.orchestra.run.smithi032.stdout:4892 writesame 0x341f4d3 thru 0x3428d2f data_size 0x1d(0x985d bytes) 2024-09-13T09:18:47.862 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:18:47.862 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:18:47.862 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-13T09:18:47.862 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:18:47.862 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:18:47.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-13T09:18:47.874 INFO:teuthology.orchestra.run.smithi032.stdout:4893 write 0xa718c15 thru 0xa727c11 (0xeffd bytes) 2024-09-13T09:18:47.880 INFO:teuthology.orchestra.run.smithi032.stdout:4894 writesame 0x1bd85dd thru 0x1be5298 data_size 0x2(0xccbc bytes) 2024-09-13T09:18:47.885 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:47.882+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:47.904 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:47.904 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:47.915 INFO:teuthology.orchestra.run.smithi032.stdout:4900 writesame 0xab0f135 thru 0xab1cb1a data_size 0x1b(0xd9e6 bytes) 2024-09-13T09:18:47.919 INFO:teuthology.orchestra.run.smithi032.stdout:4904 trunc from 0xab1cb1b to 0x89c9e64 2024-09-13T09:18:47.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:47.936 INFO:teuthology.orchestra.run.smithi032.stdout:4905 write 0xab0ac7 thru 0xab11c2 (0x6fc bytes) 2024-09-13T09:18:47.948 INFO:teuthology.orchestra.run.smithi032.stdout:4907 read 0x6d2fc10 thru 0x6d3cc3f (0xd030 bytes) 2024-09-13T09:18:47.964 INFO:teuthology.orchestra.run.smithi032.stdout:4908 writesame 0xaaa8fff thru 0xaab029c data_size 0x11(0x729e bytes) 2024-09-13T09:18:47.964 INFO:teuthology.orchestra.run.smithi032.stdout:4909 punch from 0x7157e1a to 0x715c8e3, (0x4ac9 bytes) 2024-09-13T09:18:47.964 INFO:teuthology.orchestra.run.smithi032.stdout:4910 read 0x2bfa487 thru 0x2c0826d (0xdde7 bytes) 2024-09-13T09:18:47.964 INFO:teuthology.orchestra.run.smithi032.stdout:4911 read 0x712e777 thru 0x713dcfa (0xf584 bytes) 2024-09-13T09:18:47.964 INFO:teuthology.orchestra.run.smithi032.stdout:4912 read 0x1d5a517 thru 0x1d60f00 (0x69ea bytes) 2024-09-13T09:18:47.964 INFO:teuthology.orchestra.run.smithi032.stdout:4913 read 0x7bdfc78 thru 0x7be5d24 (0x60ad bytes) 2024-09-13T09:18:47.964 INFO:teuthology.orchestra.run.smithi032.stdout:4914 punch from 0x6b3be8e to 0x6b41e14, (0x5f86 bytes) 2024-09-13T09:18:47.964 INFO:teuthology.orchestra.run.smithi032.stdout:4915 writesame 0x843f310 thru 0x844f25a data_size 0x5(0xff4b bytes) 2024-09-13T09:18:47.987 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:47.986+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:47.988 INFO:teuthology.orchestra.run.smithi032.stdout:4918 trunc from 0xaab029d to 0x2a430be 2024-09-13T09:18:47.995 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "" 2024-09-13T09:18:47.995 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:48.023 INFO:teuthology.orchestra.run.smithi032.stdout:4921 read 0x29992d0 thru 0x29a4d00 (0xba31 bytes) 2024-09-13T09:18:48.024 INFO:teuthology.orchestra.run.smithi032.stdout:4922 read 0x3d77f6 thru 0x3d8547 (0xd52 bytes) 2024-09-13T09:18:48.052 INFO:teuthology.orchestra.run.smithi032.stdout:4923 write 0xd7c74e6 thru 0xd7d4390 (0xceab bytes) 2024-09-13T09:18:48.057 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:48.057 INFO:teuthology.orchestra.run.smithi032.stdout:4925 read 0xa45de12 thru 0xa46be3f (0xe02e bytes) 2024-09-13T09:18:48.057 INFO:teuthology.orchestra.run.smithi032.stdout:4926 writesame 0xa2cf815 thru 0xa2d48c6 data_size 0x21(0x50b2 bytes) 2024-09-13T09:18:48.063 INFO:teuthology.orchestra.run.smithi032.stdout:4927 read 0x28968de thru 0x28a103f (0xa762 bytes) 2024-09-13T09:18:48.064 INFO:teuthology.orchestra.run.smithi032.stdout:4928 punch from 0x95f9457 to 0x96013eb, (0x7f94 bytes) 2024-09-13T09:18:48.064 INFO:teuthology.orchestra.run.smithi032.stdout:4930 punch from 0x1d5f2db to 0x1d6c6d9, (0xd3fe bytes) 2024-09-13T09:18:48.064 INFO:teuthology.orchestra.run.smithi032.stdout:4931 read 0x50d1c81 thru 0x50d57b4 (0x3b34 bytes) 2024-09-13T09:18:48.076 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:48.076 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:48.084 INFO:teuthology.orchestra.run.smithi032.stdout:4932 trunc from 0xd7d4391 to 0x90c9cb 2024-09-13T09:18:48.088 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:48.086+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:48.098 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:48.115 INFO:teuthology.orchestra.run.smithi032.stdout:4936 read 0x2d3386 thru 0x2db299 (0x7f14 bytes) 2024-09-13T09:18:48.138 INFO:teuthology.orchestra.run.smithi032.stdout:4937 writesame 0x6d6c368 thru 0x6d700fd data_size 0x1(0x3d96 bytes) 2024-09-13T09:18:48.138 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:48.138 INFO:teuthology.orchestra.run.smithi032.stdout:4941 clone 34 order 23 su 524288 sc 3 2024-09-13T09:18:48.189 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:48.182+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:48.242 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:48.242 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:48.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:48.299 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:48.298+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:48.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:48.423 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:48.422+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:48.426 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:48.426 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:48.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:48.492 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:48.486+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:48.525 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:48.527 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:18:48.544 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:48.563 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:48.563 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:48.580 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:48.581 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:48.581 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:48.594 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:48.590+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:48.598 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:48.600 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:48.600 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:48.617 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:48.618 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:48.620 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone33 from 0x6d700fe (overlap 0x90c9cb) to 0x824f36 2024-09-13T09:18:48.636 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:48.654 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:48.654 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-13T09:18:48.696 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:48.693+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:48.785 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:48.785 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:48.785 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:48.785 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:48.785 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:48.798 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:48.793+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:48.800 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:48.899 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:48.893+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:48.931 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4388 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.942693399000007, 2024-09-13T09:18:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069349509000000004, 2024-09-13T09:18:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4388 2024-09-13T09:18:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985769+0000", 2024-09-13T09:18:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985775+0000", 2024-09-13T09:18:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5230000000000001e-06 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985794+0000", 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8425999999999999e-05 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985809+0000", 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.486e-05 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.054965+0000", 2024-09-13T09:18:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069156463000000001 2024-09-13T09:18:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055036+0000", 2024-09-13T09:18:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0780999999999999e-05 2024-09-13T09:18:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055120+0000", 2024-09-13T09:18:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3486000000000007e-05 2024-09-13T09:18:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4389 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.942642875999994, 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069473833999999998, 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4389 2024-09-13T09:18:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985820+0000", 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985859+0000", 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9279999999999997e-05 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985881+0000", 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2116999999999998e-05 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985896+0000", 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4698999999999999e-05 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055152+0000", 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069256568000000004 2024-09-13T09:18:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055220+0000", 2024-09-13T09:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8255000000000003e-05 2024-09-13T09:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055294+0000", 2024-09-13T09:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3858999999999999e-05 2024-09-13T09:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4390 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.942547263999998, 2024-09-13T09:18:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069563635999999998, 2024-09-13T09:18:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4390 2024-09-13T09:18:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985915+0000", 2024-09-13T09:18:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985922+0000", 2024-09-13T09:18:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:18:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985941+0000", 2024-09-13T09:18:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8260000000000001e-05 2024-09-13T09:18:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985955+0000", 2024-09-13T09:18:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4409999999999999e-05 2024-09-13T09:18:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055325+0000", 2024-09-13T09:18:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069370290000000001 2024-09-13T09:18:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055395+0000", 2024-09-13T09:18:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0158000000000002e-05 2024-09-13T09:18:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055480+0000", 2024-09-13T09:18:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4340999999999998e-05 2024-09-13T09:18:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.942496196999997, 2024-09-13T09:18:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:18:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:18:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:18:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:18:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:18:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.942447461, 2024-09-13T09:18:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:18:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:18:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:18:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:18:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:18:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.942379791999997, 2024-09-13T09:18:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:18:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:18:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:18:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:18:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:18:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:18:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:18:48.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.942239583000003, 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.962 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:18:48.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:18:48.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:18:48.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:18:48.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.966 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.942148716999995, 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.967 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:18:48.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:18:48.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:18:48.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.971 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.972 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:48.972 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.942073856999997, 2024-09-13T09:18:48.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:18:48.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:18:48.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.972 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:18:48.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:18:48.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:18:48.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:18:48.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:18:48.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:18:48.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:18:48.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:18:48.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:18:48.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.976 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.942002681000005, 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:18:48.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:18:48.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:18:48.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:18:48.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:18:48.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:18:48.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:18:48.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:18:48.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:18:48.981 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.981 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.981 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.981 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.941953627999993, 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:18:48.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:18:48.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:18:48.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:18:48.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.941891892000001, 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.986 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:18:48.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.941842222000005, 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.991 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:18:48.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:18:48.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:18:48.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:18:48.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:18:48.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:18:48.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:48.995 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.941793703000002, 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:48.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:48.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:18:48.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:18:48.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:48.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:48.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:48.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:18:49.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:18:49.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:18:49.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:18:49.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.003 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.004 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:49.004 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:49.004 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.941745307000005, 2024-09-13T09:18:49.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:18:49.004 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.004 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.004 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.004 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.004 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:18:49.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:49.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:18:49.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:18:49.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:18:49.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:18:49.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:18:49.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:18:49.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.009 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.010 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:49.010 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:49.010 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.941698020000004, 2024-09-13T09:18:49.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:18:49.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.011 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.011 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:18:49.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.012 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:49.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:18:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:18:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:18:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:18:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:18:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:18:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:18:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:18:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:18:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:18:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:18:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:18:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.941614618000003, 2024-09-13T09:18:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:18:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:18:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:18:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:18:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:18:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:18:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:18:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.941540892000006, 2024-09-13T09:18:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:18:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:18:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:18:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:18:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:18:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:18:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:18:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.941472769000001, 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:18:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:18:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:18:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:18:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:18:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:18:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:18:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.941422302000007, 2024-09-13T09:18:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:18:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:18:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:18:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:18:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:18:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:18:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:18:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:18:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.040 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:49.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:49.042 INFO:teuthology.orchestra.run.smithi032.stdout:4943 read 0x163138b thru 0x163eada (0xd750 bytes) 2024-09-13T09:18:49.043 INFO:teuthology.orchestra.run.smithi032.stdout:4944 read 0x599bf06 thru 0x59a2b1e (0x6c19 bytes) 2024-09-13T09:18:49.043 INFO:teuthology.orchestra.run.smithi032.stdout:4945 write 0x985dc3 thru 0x98efb9 (0x91f7 bytes) 2024-09-13T09:18:49.043 INFO:teuthology.orchestra.run.smithi032.stdout:4951 punch from 0x24a19b5 to 0x24b0808, (0xee53 bytes) 2024-09-13T09:18:49.043 INFO:teuthology.orchestra.run.smithi032.stdout:4952 punch from 0x63614a9 to 0x63703e8, (0xef3f bytes) 2024-09-13T09:18:49.043 INFO:teuthology.orchestra.run.smithi032.stdout:4953 trunc from 0x6d700fe to 0xde7a471 2024-09-13T09:18:49.043 INFO:teuthology.orchestra.run.smithi032.stdout:4954 trunc from 0xde7a471 to 0xce94d66 2024-09-13T09:18:49.043 INFO:teuthology.orchestra.run.smithi032.stdout:4955 writesame 0xa835f2e thru 0xa83cc09 data_size 0x2(0x6cdc bytes) 2024-09-13T09:18:49.043 INFO:teuthology.orchestra.run.smithi032.stdout:4958 writesame 0xcb569df thru 0xcb5a8fe data_size 0x50(0x3f20 bytes) 2024-09-13T09:18:49.043 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:49.037+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:49.057 INFO:teuthology.orchestra.run.smithi032.stdout:4960 trunc from 0xce94d66 to 0x561b535 2024-09-13T09:18:49.084 INFO:teuthology.orchestra.run.smithi032.stdout:4963 punch from 0x7bd9e to 0x8a61e, (0xe880 bytes) 2024-09-13T09:18:49.093 INFO:teuthology.orchestra.run.smithi032.stdout:4964 trunc from 0x561b535 to 0x9be70eb 2024-09-13T09:18:49.103 INFO:teuthology.orchestra.run.smithi032.stdout:4965 read 0x5cd6ce2 thru 0x5ce204a (0xb369 bytes) 2024-09-13T09:18:49.104 INFO:teuthology.orchestra.run.smithi032.stdout:4966 read 0x60f28c1 thru 0x60ff6fb (0xce3b bytes) 2024-09-13T09:18:49.105 INFO:teuthology.orchestra.run.smithi032.stdout:4967 write 0x597c5e2 thru 0x5989c6c (0xd68b bytes) 2024-09-13T09:18:49.109 INFO:teuthology.orchestra.run.smithi032.stdout:4968 read 0x1daaad7 thru 0x1dba648 (0xfb72 bytes) 2024-09-13T09:18:49.109 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:49.109 INFO:teuthology.orchestra.run.smithi032.stdout:4970 read 0x7e62f45 thru 0x7e6a432 (0x74ee bytes) 2024-09-13T09:18:49.129 INFO:teuthology.orchestra.run.smithi032.stdout:4973 trunc from 0x9be70eb to 0x109553b 2024-09-13T09:18:49.142 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:49.137+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:49.190 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:49.191 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:49.191 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:49.191 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:49.205 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:49.218 INFO:teuthology.orchestra.run.smithi032.stdout:4974 trunc from 0x109553b to 0xae457c2 2024-09-13T09:18:49.228 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:49.228 INFO:teuthology.orchestra.run.smithi032.stdout:4976 write 0xa55e3a5 thru 0xa5635ab (0x5207 bytes) 2024-09-13T09:18:49.244 INFO:teuthology.orchestra.run.smithi032.stdout:4977 trunc from 0xae457c2 to 0x66082d3 2024-09-13T09:18:49.249 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:49.246+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:49.271 INFO:teuthology.orchestra.run.smithi032.stdout:4978 trunc from 0x66082d3 to 0xad49f01 2024-09-13T09:18:49.281 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:49.283 INFO:teuthology.orchestra.run.smithi032.stdout:4982 trunc from 0xad49f01 to 0xb63680d 2024-09-13T09:18:49.294 INFO:teuthology.orchestra.run.smithi032.stdout:4983 read 0x83c34b1 thru 0x83cf945 (0xc495 bytes) 2024-09-13T09:18:49.294 INFO:teuthology.orchestra.run.smithi032.stdout:4984 read 0x682630b thru 0x682c086 (0x5d7c bytes) 2024-09-13T09:18:49.294 INFO:teuthology.orchestra.run.smithi032.stdout:4987 punch from 0xa2174ef to 0xa2183a6, (0xeb7 bytes) 2024-09-13T09:18:49.314 INFO:teuthology.orchestra.run.smithi032.stdout:4988 write 0xcaf233d thru 0xcafd94e (0xb612 bytes) 2024-09-13T09:18:49.318 INFO:teuthology.orchestra.run.smithi032.stdout:4989 punch from 0x672bb99 to 0x672ec3d, (0x30a4 bytes) 2024-09-13T09:18:49.318 INFO:teuthology.orchestra.run.smithi032.stdout:4990 punch from 0x7263acd to 0x726c3ab, (0x88de bytes) 2024-09-13T09:18:49.318 INFO:teuthology.orchestra.run.smithi032.stdout:4991 punch from 0x29d0118 to 0x29d459d, (0x4485 bytes) 2024-09-13T09:18:49.319 INFO:teuthology.orchestra.run.smithi032.stdout:4993 write 0x7b91107 thru 0x7b942c2 (0x31bc bytes) 2024-09-13T09:18:49.325 INFO:teuthology.orchestra.run.smithi032.stdout:4996 punch from 0x84f9a63 to 0x84fb37d, (0x191a bytes) 2024-09-13T09:18:49.326 INFO:teuthology.orchestra.run.smithi032.stdout:4997 write 0x1bdf238 thru 0x1be1bbb (0x2984 bytes) 2024-09-13T09:18:49.332 INFO:teuthology.orchestra.run.smithi032.stdout:4998 clone 35 order 20 su 524288 sc 9 2024-09-13T09:18:49.342 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:49.343 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:49.343 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:49.338+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:49.357 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:49.444 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:49.441+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:49.525 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:18:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 80.836819699000003, 2024-09-13T09:18:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.21372158699999999, 2024-09-13T09:18:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685363+0000", 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685372+0000", 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4590000000000004e-06 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685394+0000", 2024-09-13T09:18:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1616000000000001e-05 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685407+0000", 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3675e-05 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685484+0000", 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7065999999999997e-05 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685493+0000", 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8070000000000006e-06 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690770+0000", 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052763869999999996 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690940+0000", 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017001900000000001 2024-09-13T09:18:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.691238+0000", 2024-09-13T09:18:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029794499999999998 2024-09-13T09:18:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.692057+0000", 2024-09-13T09:18:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081943000000000001 2024-09-13T09:18:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.898933+0000", 2024-09-13T09:18:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.20687588200000001 2024-09-13T09:18:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899026+0000", 2024-09-13T09:18:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2777999999999998e-05 2024-09-13T09:18:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899086+0000", 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0188e-05 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:14590 3.1d 3:be60f898:::rbd_data.11853ecd5f5f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13672152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 61.980971259, 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052557382, 2024-09-13T09:18:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 14590 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541207+0000", 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562477+0000", 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021270155999999998 2024-09-13T09:18:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562539+0000", 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1851000000000001e-05 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562565+0000", 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6194e-05 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562702+0000", 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013731299999999999 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.563071+0000", 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036892899999999999 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.565167+0000", 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020961959999999998 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.585261+0000", 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020093443999999999 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593596+0000", 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083352849999999996 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593696+0000", 2024-09-13T09:18:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100021 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593770+0000", 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3889999999999999e-05 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:14584 3.2 e39/22 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head v 39'64, mlcod=38'62)", 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 61.978985612999999, 2024-09-13T09:18:49.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051546091000000002, 2024-09-13T09:18:49.540 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.540 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:49.540 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:49.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:49.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543197+0000", 2024-09-13T09:18:49.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:49.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588691+0000", 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045493475999999998 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588711+0000", 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0485000000000001e-05 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588721+0000", 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.499e-06 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588766+0000", 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5228000000000001e-05 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588788+0000", 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2503000000000001e-05 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594671+0000", 2024-09-13T09:18:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058822809999999996 2024-09-13T09:18:49.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594744+0000", 2024-09-13T09:18:49.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3776000000000007e-05 2024-09-13T09:18:49.544 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.544 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.544 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.544 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:22580 3.1c 3:39876c9f:::rbd_data.1185d6f8ec87.000000000000000f:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T09:18:49.544 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:49.545 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.669732822999997, 2024-09-13T09:18:49.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050372896, 2024-09-13T09:18:49.545 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.545 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:49.545 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.545 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.545 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.545 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22580 2024-09-13T09:18:49.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.545 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852449+0000", 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852462+0000", 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3756e-05 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852495+0000", 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2435999999999999e-05 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852516+0000", 2024-09-13T09:18:49.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1347000000000001e-05 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852644+0000", 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012777200000000001 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852784+0000", 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013980500000000001 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.860936+0000", 2024-09-13T09:18:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081526859999999993 2024-09-13T09:18:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902522+0000", 2024-09-13T09:18:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041585203000000001 2024-09-13T09:18:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902560+0000", 2024-09-13T09:18:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8603000000000003e-05 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902708+0000", 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014740900000000001 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902824+0000", 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116519 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54746 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8011776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.290802347, 2024-09-13T09:18:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041426971999999999, 2024-09-13T09:18:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54746 2024-09-13T09:18:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231381+0000", 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231384+0000", 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.599e-06 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231390+0000", 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231395+0000", 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0170000000000004e-06 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272716+0000", 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041321670999999997 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272767+0000", 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0447e-05 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272809+0000", 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1659999999999998e-05 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54986 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6348800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279893747999999, 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041375935000000003, 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54986 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242292+0000", 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3039999999999999e-06 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242298+0000", 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242303+0000", 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8359999999999999e-06 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283539+0000", 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041235330000000001 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283593+0000", 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4926000000000002e-05 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283666+0000", 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2718000000000004e-05 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54987 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6447104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279876566, 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041604369000000002, 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54987 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242309+0000", 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1160000000000002e-06 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242317+0000", 2024-09-13T09:18:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9799999999999998e-06 2024-09-13T09:18:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242323+0000", 2024-09-13T09:18:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6439999999999997e-06 2024-09-13T09:18:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283711+0000", 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388085999999998 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283766+0000", 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5112999999999999e-05 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283912+0000", 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014575300000000001 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54988 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6545408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279856746, 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041865404000000002, 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54988 2024-09-13T09:18:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242329+0000", 2024-09-13T09:18:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:18:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242335+0000", 2024-09-13T09:18:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242341+0000", 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9440000000000002e-06 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283952+0000", 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041610981999999998 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284076+0000", 2024-09-13T09:18:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124356 2024-09-13T09:18:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284193+0000", 2024-09-13T09:18:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116345 2024-09-13T09:18:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54989 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6643712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279838632000001, 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042123657000000002, 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54989 2024-09-13T09:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242348+0000", 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.655e-06 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242354+0000", 2024-09-13T09:18:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2269999999999998e-06 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242360+0000", 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6170000000000003e-06 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284248+0000", 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041887830000000001 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284340+0000", 2024-09-13T09:18:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1358000000000002e-05 2024-09-13T09:18:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284469+0000", 2024-09-13T09:18:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012932300000000001 2024-09-13T09:18:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54990 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6742016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279805342, 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042278333000000001, 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54990 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242378+0000", 2024-09-13T09:18:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242381+0000", 2024-09-13T09:18:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0680000000000001e-06 2024-09-13T09:18:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242392+0000", 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0762e-05 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242398+0000", 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1040000000000003e-06 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284506+0000", 2024-09-13T09:18:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042107983000000002 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284564+0000", 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.821e-05 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284657+0000", 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2588000000000002e-05 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54991 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6840320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279780551, 2024-09-13T09:18:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042489459, 2024-09-13T09:18:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54991 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242406+0000", 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242412+0000", 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5919999999999997e-06 2024-09-13T09:18:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242425+0000", 2024-09-13T09:18:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2713e-05 2024-09-13T09:18:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284694+0000", 2024-09-13T09:18:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042269558999999998 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284784+0000", 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9999000000000003e-05 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284893+0000", 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108422 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54992 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6938624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279754179999999, 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042636897999999999, 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54992 2024-09-13T09:18:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242429+0000", 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242432+0000", 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7599999999999998e-06 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242439+0000", 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5760000000000002e-06 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242448+0000", 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284935+0000", 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042486949000000003 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284991+0000", 2024-09-13T09:18:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6471000000000002e-05 2024-09-13T09:18:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285067+0000", 2024-09-13T09:18:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5543000000000005e-05 2024-09-13T09:18:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54993 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7036928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279731303, 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042867959999999997, 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54993 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242452+0000", 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242455+0000", 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4360000000000001e-06 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242463+0000", 2024-09-13T09:18:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8220000000000007e-06 2024-09-13T09:18:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242468+0000", 2024-09-13T09:18:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9479999999999999e-06 2024-09-13T09:18:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285107+0000", 2024-09-13T09:18:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042639415999999999 2024-09-13T09:18:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285196+0000", 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8721000000000007e-05 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285321+0000", 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012495 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54994 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7135232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279704139, 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043009622999999997, 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54994 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242482+0000", 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242486+0000", 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1049999999999997e-06 2024-09-13T09:18:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242489+0000", 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.004e-06 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285354+0000", 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.04286479 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285413+0000", 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9778000000000003e-05 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285489+0000", 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6141000000000002e-05 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54995 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7233536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279692768, 2024-09-13T09:18:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043166641999999998, 2024-09-13T09:18:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54995 2024-09-13T09:18:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242492+0000", 2024-09-13T09:18:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.437e-06 2024-09-13T09:18:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242496+0000", 2024-09-13T09:18:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:18:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242498+0000", 2024-09-13T09:18:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5950000000000001e-06 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285528+0000", 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043029755000000003 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285585+0000", 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285658+0000", 2024-09-13T09:18:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2713999999999995e-05 2024-09-13T09:18:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54996 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7331840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279683453000001, 2024-09-13T09:18:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043333782000000001, 2024-09-13T09:18:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54996 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242500+0000", 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242502+0000", 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3599999999999999e-06 2024-09-13T09:18:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242505+0000", 2024-09-13T09:18:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1439999999999999e-06 2024-09-13T09:18:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242507+0000", 2024-09-13T09:18:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.605e-06 2024-09-13T09:18:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285698+0000", 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043190911999999998 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285754+0000", 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5541999999999998e-05 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285834+0000", 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0426000000000006e-05 2024-09-13T09:18:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54997 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279674364, 2024-09-13T09:18:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043496174999999998, 2024-09-13T09:18:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54997 2024-09-13T09:18:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242509+0000", 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242511+0000", 2024-09-13T09:18:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.313e-06 2024-09-13T09:18:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242514+0000", 2024-09-13T09:18:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:18:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242516+0000", 2024-09-13T09:18:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3999999999999999e-06 2024-09-13T09:18:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285875+0000", 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043358578000000002 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285929+0000", 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4602999999999999e-05 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286006+0000", 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6429999999999998e-05 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54998 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7528448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.279581752, 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043666469999999999, 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54998 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242518+0000", 2024-09-13T09:18:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999161 2024-09-13T09:18:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242606+0000", 2024-09-13T09:18:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7607000000000003e-05 2024-09-13T09:18:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242617+0000", 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0882e-05 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242624+0000", 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9009999999999997e-06 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286046+0000", 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043422308 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286106+0000", 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0285000000000003e-05 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286269+0000", 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000162578 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:55006 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8314880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.276193083999999, 2024-09-13T09:18:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041461771000000001, 2024-09-13T09:18:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 55006 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245993+0000", 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2869999999999999e-06 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245999+0000", 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0159999999999997e-06 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.246004+0000", 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287345+0000", 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041341217 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287398+0000", 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2840000000000002e-05 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287453+0000", 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4746000000000001e-05 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:68662 3.1d 3:be8be2fa:::rbd_data.118556027ff4.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.3651376430000006, 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.044923689000000003, 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 68662 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157045+0000", 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157052+0000", 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8750000000000002e-06 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157066+0000", 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3815000000000001e-05 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157073+0000", 2024-09-13T09:18:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.64e-06 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157145+0000", 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2517999999999999e-05 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157267+0000", 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012185599999999999 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.164012+0000", 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0067447660000000001 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201805+0000", 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037793528999999999 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201881+0000", 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5678000000000003e-05 2024-09-13T09:18:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201933+0000", 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2346999999999999e-05 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201970+0000", 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6532000000000002e-05 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:49.630 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:49.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:49.632 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:49.629+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:49.691 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone34 from 0xcafd94f (overlap 0x109553b) to 0x0 2024-09-13T09:18:49.732 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:49.726+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:49.773 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:49.773 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:49.788 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:49.834 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:49.829+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:49.853 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-13T09:18:49.913 INFO:teuthology.orchestra.run.smithi032.stdout:5000 punch from 0x1413fd4 to 0x1415c2e, (0x1c5a bytes) 2024-09-13T09:18:49.922 INFO:teuthology.orchestra.run.smithi032.stdout:5001 read 0x56a8feb thru 0x56b0de7 (0x7dfd bytes) 2024-09-13T09:18:49.922 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:49.922 INFO:teuthology.orchestra.run.smithi032.stdout:5003 punch from 0x2891a to 0x31dd9, (0x94bf bytes) 2024-09-13T09:18:49.934 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:49.930+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:49.945 INFO:teuthology.orchestra.run.smithi032.stdout:5004 trunc from 0xcafd94f to 0x41c33a1 2024-09-13T09:18:49.971 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:49.971 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:49.971 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:49.971 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:49.971 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:49.978 INFO:teuthology.orchestra.run.smithi032.stdout:5005 read 0x91a829 thru 0x924092 (0x986a bytes) 2024-09-13T09:18:49.980 INFO:teuthology.orchestra.run.smithi032.stdout:5006 clone 36 order 20 su 262144 sc 8 2024-09-13T09:18:49.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:50.074 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:50.070+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:50.136 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:50.130+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:50.160 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:50082 3.10 3:0ec43ecb:::rbd_object_map.1185172eb410:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T09:18:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.794165194000001, 2024-09-13T09:18:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80943783999999996, 2024-09-13T09:18:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 50082 2024-09-13T09:18:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362619+0000", 2024-09-13T09:18:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362632+0000", 2024-09-13T09:18:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3338e-05 2024-09-13T09:18:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362657+0000", 2024-09-13T09:18:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5207000000000001e-05 2024-09-13T09:18:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362669+0000", 2024-09-13T09:18:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1768e-05 2024-09-13T09:18:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362738+0000", 2024-09-13T09:18:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9486999999999994e-05 2024-09-13T09:18:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:18:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362748+0000", 2024-09-13T09:18:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0010000000000001e-05 2024-09-13T09:18:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169473+0000", 2024-09-13T09:18:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80672482300000004 2024-09-13T09:18:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169884+0000", 2024-09-13T09:18:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041114299999999999 2024-09-13T09:18:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.170206+0000", 2024-09-13T09:18:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032127700000000002 2024-09-13T09:18:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171827+0000", 2024-09-13T09:18:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016217880000000001 2024-09-13T09:18:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171871+0000", 2024-09-13T09:18:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:18:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171967+0000", 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5752000000000001e-05 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.172059+0000", 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2082000000000004e-05 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56772 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 4112384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.726477075, 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054477114, 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56772 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430309+0000", 2024-09-13T09:18:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430312+0000", 2024-09-13T09:18:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.255e-06 2024-09-13T09:18:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430319+0000", 2024-09-13T09:18:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4760000000000001e-06 2024-09-13T09:18:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430324+0000", 2024-09-13T09:18:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9799999999999998e-06 2024-09-13T09:18:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484634+0000", 2024-09-13T09:18:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054310031000000002 2024-09-13T09:18:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484697+0000", 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3355999999999994e-05 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484787+0000", 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9350000000000003e-05 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56785 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5390336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722693593000001, 2024-09-13T09:18:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054655007999999998, 2024-09-13T09:18:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56785 2024-09-13T09:18:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434094+0000", 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.471e-06 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434097+0000", 2024-09-13T09:18:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0740000000000001e-06 2024-09-13T09:18:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434100+0000", 2024-09-13T09:18:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7829999999999999e-06 2024-09-13T09:18:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488422+0000", 2024-09-13T09:18:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054321855000000002 2024-09-13T09:18:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488563+0000", 2024-09-13T09:18:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014154 2024-09-13T09:18:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488748+0000", 2024-09-13T09:18:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018461399999999999 2024-09-13T09:18:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56786 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722684211000001, 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054950539999999999, 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56786 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434104+0000", 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4020000000000001e-06 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434107+0000", 2024-09-13T09:18:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.072e-06 2024-09-13T09:18:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434109+0000", 2024-09-13T09:18:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.717e-06 2024-09-13T09:18:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488795+0000", 2024-09-13T09:18:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054685661000000003 2024-09-13T09:18:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488869+0000", 2024-09-13T09:18:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4265999999999997e-05 2024-09-13T09:18:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489053+0000", 2024-09-13T09:18:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018374299999999999 2024-09-13T09:18:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56787 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5586944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722673951999999, 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055115725999999997, 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56787 2024-09-13T09:18:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434112+0000", 2024-09-13T09:18:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434114+0000", 2024-09-13T09:18:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.308e-06 2024-09-13T09:18:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434117+0000", 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434119+0000", 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.588e-06 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489097+0000", 2024-09-13T09:18:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054977824000000002 2024-09-13T09:18:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489146+0000", 2024-09-13T09:18:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8503e-05 2024-09-13T09:18:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489228+0000", 2024-09-13T09:18:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2694000000000005e-05 2024-09-13T09:18:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56788 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722665061000001, 2024-09-13T09:18:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055437562000000003, 2024-09-13T09:18:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56788 2024-09-13T09:18:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434121+0000", 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434123+0000", 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2190000000000001e-06 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434126+0000", 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434128+0000", 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8250000000000001e-06 2024-09-13T09:18:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489293+0000", 2024-09-13T09:18:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055164092999999997 2024-09-13T09:18:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489428+0000", 2024-09-13T09:18:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013572599999999999 2024-09-13T09:18:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489559+0000", 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013086 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56789 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722655912, 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055674077000000002, 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56789 2024-09-13T09:18:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434132+0000", 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.302e-06 2024-09-13T09:18:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434136+0000", 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434139+0000", 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489583+0000", 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055443994000000003 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489699+0000", 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116611 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489805+0000", 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105331 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56790 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722645537, 2024-09-13T09:18:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055716839999999997, 2024-09-13T09:18:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56790 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434142+0000", 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2559999999999999e-06 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434145+0000", 2024-09-13T09:18:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.96e-06 2024-09-13T09:18:50.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434148+0000", 2024-09-13T09:18:50.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6479999999999999e-06 2024-09-13T09:18:50.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489851+0000", 2024-09-13T09:18:50.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055702792000000001 2024-09-13T09:18:50.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489858+0000", 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3900000000000004e-06 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56791 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722636765000001, 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055740029000000003, 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.205 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56791 2024-09-13T09:18:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434151+0000", 2024-09-13T09:18:50.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:18:50.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434154+0000", 2024-09-13T09:18:50.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0139999999999999e-06 2024-09-13T09:18:50.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434157+0000", 2024-09-13T09:18:50.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6699999999999998e-06 2024-09-13T09:18:50.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489884+0000", 2024-09-13T09:18:50.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055727511 2024-09-13T09:18:50.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489890+0000", 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7200000000000003e-06 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56792 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722627836000001, 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055760363, 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.210 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56792 2024-09-13T09:18:50.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.211 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:50.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:50.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:50.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434163+0000", 2024-09-13T09:18:50.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-13T09:18:50.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434167+0000", 2024-09-13T09:18:50.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.292e-06 2024-09-13T09:18:50.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434169+0000", 2024-09-13T09:18:50.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:18:50.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489914+0000", 2024-09-13T09:18:50.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055744415999999998 2024-09-13T09:18:50.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489919+0000", 2024-09-13T09:18:50.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:18:50.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.215 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.216 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56793 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.216 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:50.216 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722614316, 2024-09-13T09:18:50.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055774049999999999, 2024-09-13T09:18:50.216 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.216 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56793 2024-09-13T09:18:50.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.216 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:50.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:50.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:50.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434173+0000", 2024-09-13T09:18:50.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-06 2024-09-13T09:18:50.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434177+0000", 2024-09-13T09:18:50.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1070000000000001e-06 2024-09-13T09:18:50.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434179+0000", 2024-09-13T09:18:50.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.666e-06 2024-09-13T09:18:50.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489941+0000", 2024-09-13T09:18:50.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055762071000000003 2024-09-13T09:18:50.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489946+0000", 2024-09-13T09:18:50.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:18:50.221 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.221 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.221 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.221 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56794 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.221 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:50.221 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722605208000001, 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055791704999999997, 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56794 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:50.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434182+0000", 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1629999999999999e-06 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434185+0000", 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9950000000000001e-06 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434188+0000", 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489968+0000", 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055779789000000003 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489973+0000", 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2800000000000003e-06 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56795 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722596272000001, 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055809612000000001, 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56795 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434191+0000", 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.212e-06 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434194+0000", 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.078e-06 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434198+0000", 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9820000000000002e-06 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489995+0000", 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055796363000000002 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490000+0000", 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1839999999999998e-06 2024-09-13T09:18:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56796 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722586011000001, 2024-09-13T09:18:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055825859999999998, 2024-09-13T09:18:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56796 2024-09-13T09:18:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434200+0000", 2024-09-13T09:18:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434202+0000", 2024-09-13T09:18:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.204e-06 2024-09-13T09:18:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434205+0000", 2024-09-13T09:18:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9689999999999999e-06 2024-09-13T09:18:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434207+0000", 2024-09-13T09:18:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-13T09:18:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490021+0000", 2024-09-13T09:18:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055813929999999998 2024-09-13T09:18:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490027+0000", 2024-09-13T09:18:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.259e-06 2024-09-13T09:18:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56797 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.722577292, 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055843649000000002, 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56797 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434210+0000", 2024-09-13T09:18:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2389999999999999e-06 2024-09-13T09:18:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434213+0000", 2024-09-13T09:18:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:18:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434216+0000", 2024-09-13T09:18:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7800000000000001e-06 2024-09-13T09:18:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490048+0000", 2024-09-13T09:18:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055831981000000003 2024-09-13T09:18:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490053+0000", 2024-09-13T09:18:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8640000000000004e-06 2024-09-13T09:18:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56807 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.720897133999999, 2024-09-13T09:18:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054407720999999999, 2024-09-13T09:18:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56807 2024-09-13T09:18:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435889+0000", 2024-09-13T09:18:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435891+0000", 2024-09-13T09:18:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.217e-06 2024-09-13T09:18:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435894+0000", 2024-09-13T09:18:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.067e-06 2024-09-13T09:18:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435896+0000", 2024-09-13T09:18:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-13T09:18:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490292+0000", 2024-09-13T09:18:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054395840000000001 2024-09-13T09:18:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490297+0000", 2024-09-13T09:18:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:18:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56808 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.720888307999999, 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05442524, 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56808 2024-09-13T09:18:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435900+0000", 2024-09-13T09:18:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:18:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435903+0000", 2024-09-13T09:18:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:18:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435905+0000", 2024-09-13T09:18:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:18:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490319+0000", 2024-09-13T09:18:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054413396000000003 2024-09-13T09:18:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490324+0000", 2024-09-13T09:18:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:18:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56809 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.720878252, 2024-09-13T09:18:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054442786, 2024-09-13T09:18:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56809 2024-09-13T09:18:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435910+0000", 2024-09-13T09:18:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:18:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435913+0000", 2024-09-13T09:18:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1470000000000001e-06 2024-09-13T09:18:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435915+0000", 2024-09-13T09:18:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6769999999999999e-06 2024-09-13T09:18:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490346+0000", 2024-09-13T09:18:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054430574000000002 2024-09-13T09:18:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490351+0000", 2024-09-13T09:18:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2700000000000004e-06 2024-09-13T09:18:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56810 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.720868953, 2024-09-13T09:18:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054459855000000001, 2024-09-13T09:18:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56810 2024-09-13T09:18:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435917+0000", 2024-09-13T09:18:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435919+0000", 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.457e-06 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435924+0000", 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435929+0000", 2024-09-13T09:18:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8300000000000003e-06 2024-09-13T09:18:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490373+0000", 2024-09-13T09:18:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054443841999999999 2024-09-13T09:18:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490378+0000", 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8509999999999998e-06 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56811 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.720839851999999, 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054456683999999998, 2024-09-13T09:18:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56811 2024-09-13T09:18:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435946+0000", 2024-09-13T09:18:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435949+0000", 2024-09-13T09:18:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:18:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435956+0000", 2024-09-13T09:18:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8719999999999996e-06 2024-09-13T09:18:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435962+0000", 2024-09-13T09:18:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000004e-06 2024-09-13T09:18:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490398+0000", 2024-09-13T09:18:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054436824000000002 2024-09-13T09:18:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490403+0000", 2024-09-13T09:18:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0710000000000002e-06 2024-09-13T09:18:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.282 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:50.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:50.284 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:50.282+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:50.385 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:50.381+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:50.459 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:50.459 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:50.459 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:50.459 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:50.473 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:50.486 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:50.485+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:50.551 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:18:50.595 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:50.595 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:50.599 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:50.597+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:50.610 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:50.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:18:50.693 INFO:teuthology.orchestra.run.smithi032.stdout:leaving image image_client.0-clone35 intact 2024-09-13T09:18:50.694 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:50.689+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:50.751 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:18:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:18:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 99.354609120000006, 2024-09-13T09:18:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077478195999999999, 2024-09-13T09:18:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:18:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:18:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:18:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:18:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:18:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392636+0000", 2024-09-13T09:18:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392651+0000", 2024-09-13T09:18:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392674+0000", 2024-09-13T09:18:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.264e-05 2024-09-13T09:18:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392691+0000", 2024-09-13T09:18:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7203000000000001e-05 2024-09-13T09:18:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392780+0000", 2024-09-13T09:18:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8671999999999994e-05 2024-09-13T09:18:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:18:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392794+0000", 2024-09-13T09:18:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-13T09:18:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.466664+0000", 2024-09-13T09:18:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073869913999999995 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467139+0000", 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047488599999999999 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467664+0000", 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052530600000000002 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469125+0000", 2024-09-13T09:18:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001460432 2024-09-13T09:18:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469847+0000", 2024-09-13T09:18:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072197799999999996 2024-09-13T09:18:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470006+0000", 2024-09-13T09:18:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015955900000000001 2024-09-13T09:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470116+0000", 2024-09-13T09:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109836 2024-09-13T09:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:49389 3.d 3:b0b9272f:::rbd_data.1185e728d948.0000000000000025:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-13T09:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:18:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.664290344000001, 2024-09-13T09:18:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077194925999999997, 2024-09-13T09:18:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 49389 2024-09-13T09:18:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:18:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:18:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082955+0000", 2024-09-13T09:18:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082967+0000", 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2609000000000001e-05 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082998+0000", 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0596000000000003e-05 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083013+0000", 2024-09-13T09:18:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4766999999999999e-05 2024-09-13T09:18:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083136+0000", 2024-09-13T09:18:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012295599999999999 2024-09-13T09:18:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083360+0000", 2024-09-13T09:18:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000224661 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.097720+0000", 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014359627 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.159975+0000", 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062254728000000002 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.159999+0000", 2024-09-13T09:18:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.427e-05 2024-09-13T09:18:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.160089+0000", 2024-09-13T09:18:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9895999999999994e-05 2024-09-13T09:18:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.160152+0000", 2024-09-13T09:18:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2939000000000002e-05 2024-09-13T09:18:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52425 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6111232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805609+0000", 2024-09-13T09:18:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.941637719999999, 2024-09-13T09:18:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064134332000000002, 2024-09-13T09:18:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52425 2024-09-13T09:18:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805609+0000", 2024-09-13T09:18:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805609+0000", 2024-09-13T09:18:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805609+0000", 2024-09-13T09:18:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805612+0000", 2024-09-13T09:18:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.27e-06 2024-09-13T09:18:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805620+0000", 2024-09-13T09:18:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:18:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805627+0000", 2024-09-13T09:18:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8669999999999996e-06 2024-09-13T09:18:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869445+0000", 2024-09-13T09:18:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063818099000000003 2024-09-13T09:18:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869583+0000", 2024-09-13T09:18:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138078 2024-09-13T09:18:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869744+0000", 2024-09-13T09:18:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160462 2024-09-13T09:18:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52426 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6209536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:18:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.941614063999999, 2024-09-13T09:18:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064408950000000006, 2024-09-13T09:18:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52426 2024-09-13T09:18:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:18:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:18:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805632+0000", 2024-09-13T09:18:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805645+0000", 2024-09-13T09:18:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2469999999999999e-05 2024-09-13T09:18:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805654+0000", 2024-09-13T09:18:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7760000000000003e-06 2024-09-13T09:18:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805661+0000", 2024-09-13T09:18:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.114e-06 2024-09-13T09:18:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869801+0000", 2024-09-13T09:18:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064139777999999995 2024-09-13T09:18:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869918+0000", 2024-09-13T09:18:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117199 2024-09-13T09:18:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870042+0000", 2024-09-13T09:18:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124196 2024-09-13T09:18:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52427 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6307840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:18:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.941580134000001, 2024-09-13T09:18:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064720860000000005, 2024-09-13T09:18:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52427 2024-09-13T09:18:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:18:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:18:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805666+0000", 2024-09-13T09:18:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805670+0000", 2024-09-13T09:18:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2200000000000001e-06 2024-09-13T09:18:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805678+0000", 2024-09-13T09:18:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.32e-06 2024-09-13T09:18:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805685+0000", 2024-09-13T09:18:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.883e-06 2024-09-13T09:18:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870103+0000", 2024-09-13T09:18:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064417962999999995 2024-09-13T09:18:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870241+0000", 2024-09-13T09:18:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138362 2024-09-13T09:18:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870388+0000", 2024-09-13T09:18:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146732 2024-09-13T09:18:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52428 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:18:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.941554404, 2024-09-13T09:18:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064990774000000001, 2024-09-13T09:18:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52428 2024-09-13T09:18:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:18:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:18:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805692+0000", 2024-09-13T09:18:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805695+0000", 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805704+0000", 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5299999999999996e-06 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805712+0000", 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870433+0000", 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064720343999999999 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870544+0000", 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011108900000000001 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870683+0000", 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000139705 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52429 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6504448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.941514108, 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065238731999999994, 2024-09-13T09:18:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52429 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805732+0000", 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805736+0000", 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8600000000000003e-06 2024-09-13T09:18:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805746+0000", 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6860000000000001e-06 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805753+0000", 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870724+0000", 2024-09-13T09:18:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064970450999999999 2024-09-13T09:18:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870836+0000", 2024-09-13T09:18:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112796 2024-09-13T09:18:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870972+0000", 2024-09-13T09:18:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135382 2024-09-13T09:18:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52430 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6602752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:18:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.941487882000001, 2024-09-13T09:18:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065499180000000004, 2024-09-13T09:18:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52430 2024-09-13T09:18:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:18:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:18:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:18:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805762+0000", 2024-09-13T09:18:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:18:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805773+0000", 2024-09-13T09:18:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1496e-05 2024-09-13T09:18:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805780+0000", 2024-09-13T09:18:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:18:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871018+0000", 2024-09-13T09:18:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065237396000000003 2024-09-13T09:18:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871124+0000", 2024-09-13T09:18:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106469 2024-09-13T09:18:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871258+0000", 2024-09-13T09:18:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134175 2024-09-13T09:18:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52431 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6701056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:18:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.941461027999999, 2024-09-13T09:18:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065755623999999999, 2024-09-13T09:18:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52431 2024-09-13T09:18:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:18:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:18:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805785+0000", 2024-09-13T09:18:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805789+0000", 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2160000000000002e-06 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805797+0000", 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4870000000000002e-06 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805804+0000", 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:18:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871298+0000", 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065494259999999999 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871405+0000", 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106512 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871542+0000", 2024-09-13T09:18:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013683599999999999 2024-09-13T09:18:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52432 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6799360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:18:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.941396329, 2024-09-13T09:18:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065976444999999995, 2024-09-13T09:18:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52432 2024-09-13T09:18:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:18:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:18:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805850+0000", 2024-09-13T09:18:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805855+0000", 2024-09-13T09:18:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8670000000000002e-06 2024-09-13T09:18:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805867+0000", 2024-09-13T09:18:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2597e-05 2024-09-13T09:18:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805876+0000", 2024-09-13T09:18:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.833e-06 2024-09-13T09:18:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871584+0000", 2024-09-13T09:18:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065707765000000001 2024-09-13T09:18:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871690+0000", 2024-09-13T09:18:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105832 2024-09-13T09:18:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871827+0000", 2024-09-13T09:18:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013744500000000001 2024-09-13T09:18:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52440 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7585792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:18:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.937272516, 2024-09-13T09:18:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064276712999999999, 2024-09-13T09:18:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52440 2024-09-13T09:18:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:18:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:18:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809974+0000", 2024-09-13T09:18:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809976+0000", 2024-09-13T09:18:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8920000000000001e-06 2024-09-13T09:18:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809981+0000", 2024-09-13T09:18:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5170000000000001e-06 2024-09-13T09:18:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809985+0000", 2024-09-13T09:18:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9539999999999998e-06 2024-09-13T09:18:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.873936+0000", 2024-09-13T09:18:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063951470999999996 2024-09-13T09:18:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874076+0000", 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014022199999999999 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874251+0000", 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017497500000000001 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52441 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7684096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.937259137, 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064567507999999996, 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52441 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809989+0000", 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8589999999999999e-06 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809994+0000", 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809999+0000", 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:18:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874317+0000", 2024-09-13T09:18:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064318742999999998 2024-09-13T09:18:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874423+0000", 2024-09-13T09:18:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106116 2024-09-13T09:18:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874555+0000", 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013196700000000001 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52442 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7782400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.937245133999999, 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064804082999999998, 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52442 2024-09-13T09:18:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:18:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810003+0000", 2024-09-13T09:18:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8360000000000001e-06 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810009+0000", 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0220000000000001e-06 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810013+0000", 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9879999999999998e-06 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874593+0000", 2024-09-13T09:18:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064579851999999993 2024-09-13T09:18:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874688+0000", 2024-09-13T09:18:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4581999999999997e-05 2024-09-13T09:18:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874806+0000", 2024-09-13T09:18:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118108 2024-09-13T09:18:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52443 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7880704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:18:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.937230238, 2024-09-13T09:18:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065175650000000002, 2024-09-13T09:18:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52443 2024-09-13T09:18:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:18:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:18:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:18:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810018+0000", 2024-09-13T09:18:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.798e-06 2024-09-13T09:18:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810023+0000", 2024-09-13T09:18:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7829999999999998e-06 2024-09-13T09:18:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810027+0000", 2024-09-13T09:18:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8890000000000004e-06 2024-09-13T09:18:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874886+0000", 2024-09-13T09:18:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064859362000000004 2024-09-13T09:18:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875029+0000", 2024-09-13T09:18:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014311900000000001 2024-09-13T09:18:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875192+0000", 2024-09-13T09:18:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000163012 2024-09-13T09:18:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52444 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7979008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:18:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.937216659000001, 2024-09-13T09:18:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065471830999999994, 2024-09-13T09:18:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52444 2024-09-13T09:18:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:18:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:18:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:18:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810032+0000", 2024-09-13T09:18:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8199999999999999e-06 2024-09-13T09:18:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810036+0000", 2024-09-13T09:18:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5510000000000001e-06 2024-09-13T09:18:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810041+0000", 2024-09-13T09:18:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0679999999999998e-06 2024-09-13T09:18:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875256+0000", 2024-09-13T09:18:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065215765999999994 2024-09-13T09:18:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875366+0000", 2024-09-13T09:18:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109639 2024-09-13T09:18:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875502+0000", 2024-09-13T09:18:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013630399999999999 2024-09-13T09:18:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52445 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:18:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.937203343, 2024-09-13T09:18:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065733503999999998, 2024-09-13T09:18:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52445 2024-09-13T09:18:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:18:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:18:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810043+0000", 2024-09-13T09:18:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810045+0000", 2024-09-13T09:18:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.813e-06 2024-09-13T09:18:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810050+0000", 2024-09-13T09:18:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5940000000000004e-06 2024-09-13T09:18:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810054+0000", 2024-09-13T09:18:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439999999999998e-06 2024-09-13T09:18:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875543+0000", 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065489393000000007 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875658+0000", 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115302 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875777+0000", 2024-09-13T09:18:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118776 2024-09-13T09:18:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52446 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8175616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:18:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.937190014, 2024-09-13T09:18:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065969204000000004, 2024-09-13T09:18:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52446 2024-09-13T09:18:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:18:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:18:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:18:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810059+0000", 2024-09-13T09:18:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.84e-06 2024-09-13T09:18:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810063+0000", 2024-09-13T09:18:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4329999999999997e-06 2024-09-13T09:18:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810067+0000", 2024-09-13T09:18:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8800000000000001e-06 2024-09-13T09:18:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875813+0000", 2024-09-13T09:18:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065745910000000005 2024-09-13T09:18:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875903+0000", 2024-09-13T09:18:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0665e-05 2024-09-13T09:18:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876026+0000", 2024-09-13T09:18:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122836 2024-09-13T09:18:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52447 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8273920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:18:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.937155773000001, 2024-09-13T09:18:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066176867, 2024-09-13T09:18:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52447 2024-09-13T09:18:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:18:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:18:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:18:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810093+0000", 2024-09-13T09:18:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6089999999999999e-06 2024-09-13T09:18:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810099+0000", 2024-09-13T09:18:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1539999999999999e-06 2024-09-13T09:18:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810104+0000", 2024-09-13T09:18:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4630000000000003e-06 2024-09-13T09:18:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876062+0000", 2024-09-13T09:18:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065957672999999994 2024-09-13T09:18:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876153+0000", 2024-09-13T09:18:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1119000000000006e-05 2024-09-13T09:18:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876268+0000", 2024-09-13T09:18:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115431 2024-09-13T09:18:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52448 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:18:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.937139605, 2024-09-13T09:18:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066534826000000005, 2024-09-13T09:18:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52448 2024-09-13T09:18:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810109+0000", 2024-09-13T09:18:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0169999999999999e-06 2024-09-13T09:18:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810115+0000", 2024-09-13T09:18:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:18:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810120+0000", 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9620000000000001e-06 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876349+0000", 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066229213999999995 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876484+0000", 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013503199999999999 2024-09-13T09:18:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876642+0000", 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158228 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52452 3.d 3:b5f40919:::rbd_data.118567e41306.0000000000000055:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.937123766999999, 2024-09-13T09:18:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070307575999999997, 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52452 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810126+0000", 2024-09-13T09:18:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:18:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810132+0000", 2024-09-13T09:18:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7349999999999999e-06 2024-09-13T09:18:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810136+0000", 2024-09-13T09:18:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2069999999999997e-06 2024-09-13T09:18:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876306+0000", 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066169943999999994 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876520+0000", 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021377500000000001 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.877399+0000", 2024-09-13T09:18:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087836900000000005 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880158+0000", 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002759657 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880212+0000", 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3551999999999998e-05 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880315+0000", 2024-09-13T09:18:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010306099999999999 2024-09-13T09:18:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880431+0000", 2024-09-13T09:18:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011603300000000001 2024-09-13T09:18:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:50.864 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:50.865 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:50.866 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:50.866 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-13T09:18:50.867 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:50.867 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:50.868 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:50.862+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:50.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:50.869 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:50.869 INFO:teuthology.orchestra.run.smithi032.stdout:5007 write 0xa0ebecf thru 0xa0f07a9 (0x48db bytes) 2024-09-13T09:18:50.876 INFO:teuthology.orchestra.run.smithi032.stdout:5008 write 0x2437b6f thru 0x244578c (0xdc1e bytes) 2024-09-13T09:18:50.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:50.886 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:50.886 INFO:teuthology.orchestra.run.smithi032.stdout:5012 read 0x687015 thru 0x68b0a5 (0x4091 bytes) 2024-09-13T09:18:50.888 INFO:teuthology.orchestra.run.smithi032.stdout:5013 punch from 0x95bb152 to 0x95c8f22, (0xddd0 bytes) 2024-09-13T09:18:50.888 INFO:teuthology.orchestra.run.smithi032.stdout:5014 punch from 0x8fb8bc2 to 0x8fbc7cc, (0x3c0a bytes) 2024-09-13T09:18:50.888 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:50.888 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:50.888 INFO:teuthology.orchestra.run.smithi032.stdout:5018 write 0x3c5f7a2 thru 0x3c69a48 (0xa2a7 bytes) 2024-09-13T09:18:50.895 INFO:teuthology.orchestra.run.smithi032.stdout:5020 clone 37 order 23 su 4194304 sc 10 2024-09-13T09:18:50.907 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:50.907 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:50.941 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:50.942 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:50.961 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:50.967 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:50.961+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:50.981 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:50.981 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:50.988 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:50.989 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:51.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:51.068 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:51.066+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:51.166 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:51.166 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:51.168 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:51.162+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:51.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:51.269 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:51.265+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:51.304 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:51.304 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:51.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:51.407 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:51.405+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:51.471 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:51.466+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:51.488 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:51.488 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:51.503 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:51.546 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:18:51.573 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:51.569+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:51.672 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:51.672 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:51.672 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:51.666+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:51.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:51.712 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone36 from 0xa0f07aa (overlap 0x41c33a1) to 0x69385c 2024-09-13T09:18:51.774 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:51.770+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:51.843 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:51.843 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:51.858 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:51.900 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:51.897+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:51.974 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:51.970+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-13T09:18:52.033 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:52.033 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:52.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:52.107 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:52.105+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:52.176 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:52.170+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:52.178 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:52.178 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:52.194 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:52.210 INFO:teuthology.orchestra.run.smithi032.stdout:5021 write 0x3526942 thru 0x3532a67 (0xc126 bytes) 2024-09-13T09:18:52.227 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:52.227 INFO:teuthology.orchestra.run.smithi032.stdout:5023 punch from 0x6caf3a4 to 0x6cba423, (0xb07f bytes) 2024-09-13T09:18:52.227 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:52.227 INFO:teuthology.orchestra.run.smithi032.stdout:5025 write 0x77586d4 thru 0x77644cd (0xbdfa bytes) 2024-09-13T09:18:52.234 INFO:teuthology.orchestra.run.smithi032.stdout:5026 write 0x131cf13 thru 0x1326037 (0x9125 bytes) 2024-09-13T09:18:52.245 INFO:teuthology.orchestra.run.smithi032.stdout:5027 read 0x832680b thru 0x832bcfa (0x54f0 bytes) 2024-09-13T09:18:52.245 INFO:teuthology.orchestra.run.smithi032.stdout:5031 punch from 0x45405f9 to 0x454be76, (0xb87d bytes) 2024-09-13T09:18:52.245 INFO:teuthology.orchestra.run.smithi032.stdout:5033 read 0x1b728c0 thru 0x1b7a72d (0x7e6e bytes) 2024-09-13T09:18:52.245 INFO:teuthology.orchestra.run.smithi032.stdout:5034 write 0x804038b thru 0x8043584 (0x31fa bytes) 2024-09-13T09:18:52.251 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xec9f4cd 2024-09-13T09:18:52.258 INFO:teuthology.orchestra.run.smithi032.stdout:5035 trunc from 0xa0f07aa to 0xec9f4cd 2024-09-13T09:18:52.266 INFO:teuthology.orchestra.run.smithi032.stdout:5036 read 0x145037c thru 0x14566be (0x6343 bytes) 2024-09-13T09:18:52.266 INFO:teuthology.orchestra.run.smithi032.stdout:5037 write 0xd79fcec thru 0xd7ab9e7 (0xbcfc bytes) 2024-09-13T09:18:52.272 INFO:teuthology.orchestra.run.smithi032.stdout:5039 writesame 0x3588c53 thru 0x3594319 data_size 0x9(0xb6c7 bytes) 2024-09-13T09:18:52.276 INFO:teuthology.orchestra.run.smithi032.stdout:5040 write 0xc1d6345 thru 0xc1e50a7 (0xed63 bytes) 2024-09-13T09:18:52.277 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:52.273+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:52.283 INFO:teuthology.orchestra.run.smithi032.stdout:5041 read 0x5cb0ced thru 0x5cb7268 (0x657c bytes) 2024-09-13T09:18:52.293 INFO:teuthology.orchestra.run.smithi032.stdout:5044 writesame 0xee50b1b thru 0xee5c214 data_size 0x25(0xb6fa bytes) 2024-09-13T09:18:52.293 INFO:teuthology.orchestra.run.smithi032.stdout:5045 writesame 0xee13136 thru 0xee1834f data_size 0x1f(0x521a bytes) 2024-09-13T09:18:52.301 INFO:teuthology.orchestra.run.smithi032.stdout:5046 write 0xe34ddc2 thru 0xe34fa24 (0x1c63 bytes) 2024-09-13T09:18:52.307 INFO:teuthology.orchestra.run.smithi032.stdout:5047 punch from 0x12a2992 to 0x12a77f9, (0x4e67 bytes) 2024-09-13T09:18:52.307 INFO:teuthology.orchestra.run.smithi032.stdout:5049 read 0xa80d0e6 thru 0xa8166f1 (0x960c bytes) 2024-09-13T09:18:52.307 INFO:teuthology.orchestra.run.smithi032.stdout:5050 punch from 0x23d3f08 to 0x23d55b8, (0x16b0 bytes) 2024-09-13T09:18:52.307 INFO:teuthology.orchestra.run.smithi032.stdout:5051 writesame 0x2a6fd88 thru 0x2a72d67 data_size 0x10(0x2fe0 bytes) 2024-09-13T09:18:52.345 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:52.345 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:52.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:52.360 INFO:teuthology.orchestra.run.smithi032.stdout:5052 write 0xe250a15 thru 0xe253019 (0x2605 bytes) 2024-09-13T09:18:52.364 INFO:teuthology.orchestra.run.smithi032.stdout:5053 punch from 0x66845a5 to 0x66905a9, (0xc004 bytes) 2024-09-13T09:18:52.364 INFO:teuthology.orchestra.run.smithi032.stdout:5054 read 0x15663fa thru 0x156ad63 (0x496a bytes) 2024-09-13T09:18:52.364 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:52.365 INFO:teuthology.orchestra.run.smithi032.stdout:5057 read 0xc84040 thru 0xc8c06e (0x802f bytes) 2024-09-13T09:18:52.365 INFO:teuthology.orchestra.run.smithi032.stdout:5058 read 0x8ca5684 thru 0x8ca93a0 (0x3d1d bytes) 2024-09-13T09:18:52.377 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:52.374+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:52.385 INFO:teuthology.orchestra.run.smithi032.stdout:5060 trunc from 0xee5c215 to 0x27b540d 2024-09-13T09:18:52.452 INFO:teuthology.orchestra.run.smithi032.stdout:5061 write 0x164d9e9 thru 0x165171c (0x3d34 bytes) 2024-09-13T09:18:52.455 INFO:teuthology.orchestra.run.smithi032.stdout:5062 write 0x9fc504 thru 0xa04fb3 (0x8ab0 bytes) 2024-09-13T09:18:52.471 INFO:teuthology.orchestra.run.smithi032.stdout:5063 trunc from 0x27b540d to 0x7e8bbe3 2024-09-13T09:18:52.478 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:52.473+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:5064 clone 38 order 24 su 524288 sc 12 2024-09-13T09:18:52.500 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:52.501 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:52.515 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:52.579 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:52.574+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:52.680 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:52.677+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:52.688 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:52.689 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:52.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:52.753 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone37 from 0x7e8bbe3 (overlap 0x27b540d) to 0x13daa06 2024-09-13T09:18:52.781 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:52.778+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:52.882 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:52.877+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:52.889 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:52.889 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:52.904 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:52.983 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:52.978+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:53.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:53.015 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:18:53.015 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:18:53.015 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-13T09:18:53.015 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:18:53.015 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:18:53.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-13T09:18:53.027 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-13T09:18:53.047 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:53.047 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:53.049 INFO:teuthology.orchestra.run.smithi032.stdout:5067 write 0x6442c10 thru 0x644af53 (0x8344 bytes) 2024-09-13T09:18:53.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:53.068 INFO:teuthology.orchestra.run.smithi032.stdout:5068 write 0x4fcc418 thru 0x4fd164d (0x5236 bytes) 2024-09-13T09:18:53.082 INFO:teuthology.orchestra.run.smithi032.stdout:5069 write 0x4c22f04 thru 0x4c2ba19 (0x8b16 bytes) 2024-09-13T09:18:53.083 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:53.081+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:53.101 INFO:teuthology.orchestra.run.smithi032.stdout:5070 trunc from 0x7e8bbe3 to 0x23ba93c 2024-09-13T09:18:53.158 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "" 2024-09-13T09:18:53.158 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:53.163 INFO:teuthology.orchestra.run.smithi032.stdout:5071 trunc from 0x23ba93c to 0xbcfcae9 2024-09-13T09:18:53.170 INFO:teuthology.orchestra.run.smithi032.stdout:5072 writesame 0xa9e325f thru 0xa9e3b84 data_size 0x1(0x926 bytes) 2024-09-13T09:18:53.173 INFO:teuthology.orchestra.run.smithi032.stdout:5073 write 0x750fb76 thru 0x7513d3a (0x41c5 bytes) 2024-09-13T09:18:53.175 INFO:teuthology.orchestra.run.smithi032.stdout:5074 write 0xa43b95e thru 0xa43e6e3 (0x2d86 bytes) 2024-09-13T09:18:53.184 INFO:teuthology.orchestra.run.smithi032.stdout:5075 writesame 0xca7cac5 thru 0xca899ca data_size 0x2(0xcf06 bytes) 2024-09-13T09:18:53.184 INFO:teuthology.orchestra.run.smithi032.stdout:5078 read 0x5468829 thru 0x54754cc (0xcca4 bytes) 2024-09-13T09:18:53.184 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:53.181+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:53.188 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:53.188 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:53.193 INFO:teuthology.orchestra.run.smithi032.stdout:5079 trunc from 0xca899cb to 0x6a4043e 2024-09-13T09:18:53.203 INFO:teuthology.orchestra.run.smithi032.stdout:5080 read 0x76c1c1 thru 0x76e994 (0x27d4 bytes) 2024-09-13T09:18:53.203 INFO:teuthology.orchestra.run.smithi032.stdout:5081 read 0x149aeda thru 0x14a64be (0xb5e5 bytes) 2024-09-13T09:18:53.204 INFO:teuthology.orchestra.run.smithi032.stdout:5082 read 0x349dbf3 thru 0x34ad753 (0xfb61 bytes) 2024-09-13T09:18:53.204 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:53.208 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:53.208 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:53.210 INFO:teuthology.orchestra.run.smithi032.stdout:5084 trunc from 0x6a4043e to 0x2a7cb35 2024-09-13T09:18:53.227 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:53.227 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:53.231 INFO:teuthology.orchestra.run.smithi032.stdout:5085 writesame 0x65c4333 thru 0x65c47c5 data_size 0x1(0x493 bytes) 2024-09-13T09:18:53.233 INFO:teuthology.orchestra.run.smithi032.stdout:5086 punch from 0x1943925 to 0x19535a6, (0xfc81 bytes) 2024-09-13T09:18:53.240 INFO:teuthology.orchestra.run.smithi032.stdout:5087 trunc from 0x65c47c6 to 0x1532f28 2024-09-13T09:18:53.244 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:53.244 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:53.246 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:53.246 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:53.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:53.263 INFO:teuthology.orchestra.run.smithi032.stdout:5088 writesame 0x43fd7bd thru 0x440aad9 data_size 0xf(0xd31d bytes) 2024-09-13T09:18:53.265 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:53.283 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:53.283 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:53.285 INFO:teuthology.orchestra.run.smithi032.stdout:5089 write 0xda2dfc9 thru 0xda346ea (0x6722 bytes) 2024-09-13T09:18:53.290 INFO:teuthology.orchestra.run.smithi032.stdout:5090 read 0x116ba21 thru 0x1178b96 (0xd176 bytes) 2024-09-13T09:18:53.291 INFO:teuthology.orchestra.run.smithi032.stdout:5091 write 0xa0d5bca thru 0xa0e3820 (0xdc57 bytes) 2024-09-13T09:18:53.293 INFO:teuthology.orchestra.run.smithi032.stdout:5092 read 0x403ef17 thru 0x4044df6 (0x5ee0 bytes) 2024-09-13T09:18:53.301 INFO:teuthology.orchestra.run.smithi032.stdout:5093 writesame 0xdd66418 thru 0xdd739c8 data_size 0x15(0xd5b1 bytes) 2024-09-13T09:18:53.301 INFO:teuthology.orchestra.run.smithi032.stdout:5096 read 0x681dc7e thru 0x682bed9 (0xe25c bytes) 2024-09-13T09:18:53.301 INFO:teuthology.orchestra.run.smithi032.stdout:5098 read 0xab31eb3 thru 0xab372aa (0x53f8 bytes) 2024-09-13T09:18:53.303 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:53.303 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:53.305 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:53.302+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:53.309 INFO:teuthology.orchestra.run.smithi032.stdout:5099 write 0xdde5d3d thru 0xddead62 (0x5026 bytes) 2024-09-13T09:18:53.313 INFO:teuthology.orchestra.run.smithi032.stdout:5100 punch from 0x593b16 to 0x5a0c25, (0xd10f bytes) 2024-09-13T09:18:53.313 INFO:teuthology.orchestra.run.smithi032.stdout:5101 read 0xa8b4561 thru 0xa8c2175 (0xdc15 bytes) 2024-09-13T09:18:53.321 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:53.322 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:53.322 INFO:teuthology.orchestra.run.smithi032.stdout:5102 write 0xe3a21fd thru 0xe3ab2c2 (0x90c6 bytes) 2024-09-13T09:18:53.324 INFO:teuthology.orchestra.run.smithi032.stdout:5103 trunc from 0xe3ab2c3 to 0xdb3b4ab 2024-09-13T09:18:53.332 INFO:teuthology.orchestra.run.smithi032.stdout:5104 write 0xb0654dd thru 0xb068fcb (0x3aef bytes) 2024-09-13T09:18:53.334 INFO:teuthology.orchestra.run.smithi032.stdout:5105 writesame 0xc11518a thru 0xc118017 data_size 0x3b(0x2e8e bytes) 2024-09-13T09:18:53.339 INFO:teuthology.orchestra.run.smithi032.stdout:5106 write 0x5b207f0 thru 0x5b300cb (0xf8dc bytes) 2024-09-13T09:18:53.362 INFO:teuthology.orchestra.run.smithi032.stdout:5107 trunc from 0xdb3b4ab to 0x95f799 2024-09-13T09:18:53.386 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:53.381+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:53.395 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:53.395 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:53.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:53.395 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:53.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4388 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.406534984000004, 2024-09-13T09:18:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069349509000000004, 2024-09-13T09:18:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4388 2024-09-13T09:18:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985770+0000", 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985769+0000", 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985775+0000", 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5230000000000001e-06 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985794+0000", 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8425999999999999e-05 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985809+0000", 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.486e-05 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.054965+0000", 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069156463000000001 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055036+0000", 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0780999999999999e-05 2024-09-13T09:18:53.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055120+0000", 2024-09-13T09:18:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3486000000000007e-05 2024-09-13T09:18:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4389 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:53.400 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.406484461000005, 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069473833999999998, 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4389 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:53.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985821+0000", 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985820+0000", 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985859+0000", 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9279999999999997e-05 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985881+0000", 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2116999999999998e-05 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985896+0000", 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4698999999999999e-05 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055152+0000", 2024-09-13T09:18:53.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069256568000000004 2024-09-13T09:18:53.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055220+0000", 2024-09-13T09:18:53.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8255000000000003e-05 2024-09-13T09:18:53.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055294+0000", 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3858999999999999e-05 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4390 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.406388848999995, 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069563635999999998, 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4390 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985916+0000", 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985915+0000", 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985922+0000", 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985941+0000", 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8260000000000001e-05 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985955+0000", 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4409999999999999e-05 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055325+0000", 2024-09-13T09:18:53.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069370290000000001 2024-09-13T09:18:53.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055395+0000", 2024-09-13T09:18:53.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0158000000000002e-05 2024-09-13T09:18:53.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055480+0000", 2024-09-13T09:18:53.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4340999999999998e-05 2024-09-13T09:18:53.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.409 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.410 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.410 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.410 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:53.410 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.406337781999994, 2024-09-13T09:18:53.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:18:53.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.411 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:18:53.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.411 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:53.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:18:53.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.406289045999998, 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:18:53.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:18:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:18:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:18:53.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:18:53.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:18:53.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:18:53.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.419 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.420 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.420 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:53.420 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.406221376999994, 2024-09-13T09:18:53.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:18:53.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.420 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:18:53.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.420 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:18:53.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.406081168, 2024-09-13T09:18:53.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:18:53.428 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405990302000006, 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.429 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:18:53.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:18:53.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:18:53.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:18:53.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:18:53.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:18:53.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:18:53.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:18:53.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:18:53.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.434 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.434 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.434 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:53.434 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405915441999994, 2024-09-13T09:18:53.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:18:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:18:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:18:53.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:18:53.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:18:53.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:18:53.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:18:53.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405844266000003, 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.439 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:18:53.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.440 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:53.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:18:53.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.443 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.444 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.444 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:53.444 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405795213000005, 2024-09-13T09:18:53.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:18:53.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:18:53.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.449 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.449 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:53.449 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405733476999998, 2024-09-13T09:18:53.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:18:53.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:18:53.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.449 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:18:53.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:53.453 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405683807000003, 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:53.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:18:53.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405635287999999, 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:18:53.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:18:53.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.463 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.463 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:53.463 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405586892000002, 2024-09-13T09:18:53.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:18:53.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.463 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:18:53.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.463 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:53.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:18:53.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:18:53.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:18:53.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:53.467 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405539605000001, 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:18:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:18:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:18:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:18:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405456203, 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.472 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:18:53.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:18:53.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:18:53.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:18:53.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:18:53.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:18:53.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:18:53.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:18:53.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:18:53.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:18:53.476 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.476 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405382477000003, 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:18:53.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:18:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:18:53.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:18:53.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:18:53.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:18:53.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.481 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.481 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405314353999998, 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:18:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:18:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:18:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.405263887000004, 2024-09-13T09:18:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:53.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:18:53.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:53.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:53.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:18:53.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:53.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:53.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:18:53.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.491 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:53.491 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:53.491 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:53.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:53.494 INFO:teuthology.orchestra.run.smithi032.stdout:5108 write 0xa92695f thru 0xa93364e (0xccf0 bytes) 2024-09-13T09:18:53.494 INFO:teuthology.orchestra.run.smithi032.stdout:5109 punch from 0x373d75e to 0x373e6f1, (0xf93 bytes) 2024-09-13T09:18:53.494 INFO:teuthology.orchestra.run.smithi032.stdout:5112 write 0x45731ea thru 0x45758be (0x26d5 bytes) 2024-09-13T09:18:53.494 INFO:teuthology.orchestra.run.smithi032.stdout:5114 trunc from 0xa93364f to 0xec19f68 2024-09-13T09:18:53.494 INFO:teuthology.orchestra.run.smithi032.stdout:5115 writesame 0xbc3b134 thru 0xbc3d0b7 data_size 0x2(0x1f84 bytes) 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:5116 punch from 0x3c69ba to 0x3cfcab, (0x92f1 bytes) 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:5117 write 0xf99f1b thru 0xf9bab2 (0x1b98 bytes) 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:5120 punch from 0xb0a3e68 to 0xb0a6fbd, (0x3155 bytes) 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:5123 read 0x260088a thru 0x26011ff (0x976 bytes) 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:5125 read 0x983da58 thru 0x984d3e5 (0xf98e bytes) 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:5126 write 0xe4b6840 thru 0xe4b7d68 (0x1529 bytes) 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:5127 read 0xa850b68 thru 0xa859536 (0x89cf bytes) 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:5130 punch from 0x1197b70 to 0x11a2cb5, (0xb145 bytes) 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:5131 writesame 0xc2cdfd5 thru 0xc2d458d data_size 0x1(0x65b9 bytes) 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:5133 read 0xea80b54 thru 0xea90055 (0xf502 bytes) 2024-09-13T09:18:53.495 INFO:teuthology.orchestra.run.smithi032.stdout:5135 writesame 0x858f9d3 thru 0x859bcc4 data_size 0x1(0xc2f2 bytes) 2024-09-13T09:18:53.496 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:53.489+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:53.535 INFO:teuthology.orchestra.run.smithi032.stdout:5136 read 0x5363f3 thru 0x545415 (0xf023 bytes) 2024-09-13T09:18:53.537 INFO:teuthology.orchestra.run.smithi032.stdout:5137 read 0x71fe673 thru 0x7208077 (0x9a05 bytes) 2024-09-13T09:18:53.537 INFO:teuthology.orchestra.run.smithi032.stdout:5138 write 0x7a63e5e thru 0x7a644e3 (0x686 bytes) 2024-09-13T09:18:53.541 INFO:teuthology.orchestra.run.smithi032.stdout:5139 write 0xa4d0d7c thru 0xa4dfc53 (0xeed8 bytes) 2024-09-13T09:18:53.546 INFO:teuthology.orchestra.run.smithi032.stdout:5140 punch from 0x5ff38bf to 0x5ff6120, (0x2861 bytes) 2024-09-13T09:18:53.546 INFO:teuthology.orchestra.run.smithi032.stdout:5142 read 0x4fd4055 thru 0x4fd63d7 (0x2383 bytes) 2024-09-13T09:18:53.546 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:53.547 INFO:teuthology.orchestra.run.smithi032.stdout:5144 write 0xb7e1bcd thru 0xb7f0a84 (0xeeb8 bytes) 2024-09-13T09:18:53.550 INFO:teuthology.orchestra.run.smithi032.stdout:5147 write 0x775fcad thru 0x77699a3 (0x9cf7 bytes) 2024-09-13T09:18:53.553 INFO:teuthology.orchestra.run.smithi032.stdout:5148 punch from 0x4241f0d to 0x424fc4a, (0xdd3d bytes) 2024-09-13T09:18:53.553 INFO:teuthology.orchestra.run.smithi032.stdout:5149 write 0x53834d4 thru 0x538e626 (0xb153 bytes) 2024-09-13T09:18:53.557 INFO:teuthology.orchestra.run.smithi032.stdout:5150 write 0x6985c9 thru 0x69f355 (0x6d8d bytes) 2024-09-13T09:18:53.574 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:18:53.577 INFO:teuthology.orchestra.run.smithi032.stdout:5151 trunc from 0xec19f68 to 0x87ac4f4 2024-09-13T09:18:53.613 INFO:teuthology.orchestra.run.smithi032.stdout:5153 trunc from 0x87ac4f4 to 0xe74aedd 2024-09-13T09:18:53.614 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:53.609+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:53.622 INFO:teuthology.orchestra.run.smithi032.stdout:5154 punch from 0x70d5800 to 0x70e2d1b, (0xd51b bytes) 2024-09-13T09:18:53.622 INFO:teuthology.orchestra.run.smithi032.stdout:5155 read 0x71a98ab thru 0x71b6571 (0xccc7 bytes) 2024-09-13T09:18:53.623 INFO:teuthology.orchestra.run.smithi032.stdout:5156 punch from 0xcd0e027 to 0xcd1b461, (0xd43a bytes) 2024-09-13T09:18:53.623 INFO:teuthology.orchestra.run.smithi032.stdout:5161 read 0x2f92644 thru 0x2f9e9ab (0xc368 bytes) 2024-09-13T09:18:53.625 INFO:teuthology.orchestra.run.smithi032.stdout:5162 punch from 0xb445242 to 0xb450b76, (0xb934 bytes) 2024-09-13T09:18:53.625 INFO:teuthology.orchestra.run.smithi032.stdout:5163 punch from 0x6e1834b to 0x6e24d1e, (0xc9d3 bytes) 2024-09-13T09:18:53.625 INFO:teuthology.orchestra.run.smithi032.stdout:5164 read 0xc084dfa thru 0xc08b68c (0x6893 bytes) 2024-09-13T09:18:53.625 INFO:teuthology.orchestra.run.smithi032.stdout:5165 writesame 0x9d1d84b thru 0x9d2b03b data_size 0x1(0xd7f1 bytes) 2024-09-13T09:18:53.662 INFO:teuthology.orchestra.run.smithi032.stdout:5167 read 0x8a6ef5e thru 0x8a7c113 (0xd1b6 bytes) 2024-09-13T09:18:53.663 INFO:teuthology.orchestra.run.smithi032.stdout:5168 write 0x17c8102 thru 0x17d2261 (0xa160 bytes) 2024-09-13T09:18:53.666 INFO:teuthology.orchestra.run.smithi032.stdout:5169 read 0x9cf93ef thru 0x9cffa92 (0x66a4 bytes) 2024-09-13T09:18:53.667 INFO:teuthology.orchestra.run.smithi032.stdout:5170 write 0x9b60e62 thru 0x9b64f28 (0x40c7 bytes) 2024-09-13T09:18:53.670 INFO:teuthology.orchestra.run.smithi032.stdout:5171 write 0x92d693e thru 0x92deecb (0x858e bytes) 2024-09-13T09:18:53.674 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:53.674 INFO:teuthology.orchestra.run.smithi032.stdout:5173 write 0x11dd169 thru 0x11e2464 (0x52fc bytes) 2024-09-13T09:18:53.676 INFO:teuthology.orchestra.run.smithi032.stdout:5175 writesame 0x371c08a thru 0x37276b3 data_size 0x7(0xb62a bytes) 2024-09-13T09:18:53.682 INFO:teuthology.orchestra.run.smithi032.stdout:5176 writesame 0xac81f9d thru 0xac91aed data_size 0xd(0xfb51 bytes) 2024-09-13T09:18:53.686 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:53.687 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:53.689 INFO:teuthology.orchestra.run.smithi032.stdout:5177 read 0x91521d7 thru 0x91588c3 (0x66ed bytes) 2024-09-13T09:18:53.690 INFO:teuthology.orchestra.run.smithi032.stdout:5179 writesame 0xbadc98a thru 0xbae714d data_size 0x12(0xa7c4 bytes) 2024-09-13T09:18:53.694 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:53.690+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:53.694 INFO:teuthology.orchestra.run.smithi032.stdout:5180 writesame 0x9af1c7b thru 0x9b01a2b data_size 0x1f(0xfdb1 bytes) 2024-09-13T09:18:53.699 INFO:teuthology.orchestra.run.smithi032.stdout:5184 writesame 0xdb146cd thru 0xdb24374 data_size 0xa8(0xfca8 bytes) 2024-09-13T09:18:53.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:53.706 INFO:teuthology.orchestra.run.smithi032.stdout:5185 read 0xc78392c thru 0xc79016a (0xc83f bytes) 2024-09-13T09:18:53.706 INFO:teuthology.orchestra.run.smithi032.stdout:5188 read 0xb089afb thru 0xb08feb2 (0x63b8 bytes) 2024-09-13T09:18:53.707 INFO:teuthology.orchestra.run.smithi032.stdout:5189 write 0x2d759e9 thru 0x2d7882b (0x2e43 bytes) 2024-09-13T09:18:53.709 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:53.709 INFO:teuthology.orchestra.run.smithi032.stdout:5193 writesame 0xa5eb86a thru 0xa5eff99 data_size 0x44(0x4730 bytes) 2024-09-13T09:18:53.711 INFO:teuthology.orchestra.run.smithi032.stdout:5194 read 0xd2a46cf thru 0xd2b09e1 (0xc313 bytes) 2024-09-13T09:18:53.711 INFO:teuthology.orchestra.run.smithi032.stdout:5195 write 0xcc99654 thru 0xcc9cc48 (0x35f5 bytes) 2024-09-13T09:18:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:5196 trunc from 0xe74aedd to 0xcd92896 2024-09-13T09:18:53.729 INFO:teuthology.orchestra.run.smithi032.stdout:5197 write 0xaabe18d thru 0xaacb6be (0xd532 bytes) 2024-09-13T09:18:53.730 INFO:teuthology.orchestra.run.smithi032.stdout:5198 write 0xc34f97d thru 0xc35e39d (0xea21 bytes) 2024-09-13T09:18:53.733 INFO:teuthology.orchestra.run.smithi032.stdout:5200 read 0xa8aac45 thru 0xa8b6231 (0xb5ed bytes) 2024-09-13T09:18:53.734 INFO:teuthology.orchestra.run.smithi032.stdout:5201 write 0x98b2b47 thru 0x98bbbcc (0x9086 bytes) 2024-09-13T09:18:53.735 INFO:teuthology.orchestra.run.smithi032.stdout:5203 write 0x477c0dc thru 0x477d34f (0x1274 bytes) 2024-09-13T09:18:53.738 INFO:teuthology.orchestra.run.smithi032.stdout:5204 read 0x598cc2a thru 0x59932ab (0x6682 bytes) 2024-09-13T09:18:53.738 INFO:teuthology.orchestra.run.smithi032.stdout:5205 write 0x1665447 thru 0x1666622 (0x11dc bytes) 2024-09-13T09:18:53.740 INFO:teuthology.orchestra.run.smithi032.stdout:5206 punch from 0xe98da7 to 0xe9b05a, (0x22b3 bytes) 2024-09-13T09:18:53.740 INFO:teuthology.orchestra.run.smithi032.stdout:5207 writesame 0x20369eb thru 0x20461c5 data_size 0x1(0xf7db bytes) 2024-09-13T09:18:53.790 INFO:teuthology.orchestra.run.smithi032.stdout:5208 trunc from 0xcd92896 to 0x6e1adfe 2024-09-13T09:18:53.795 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:53.789+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:53.812 INFO:teuthology.orchestra.run.smithi032.stdout:5209 write 0x2a53776 thru 0x2a5442a (0xcb5 bytes) 2024-09-13T09:18:53.814 INFO:teuthology.orchestra.run.smithi032.stdout:5210 punch from 0x59ddcb6 to 0x59ea57f, (0xc8c9 bytes) 2024-09-13T09:18:53.825 INFO:teuthology.orchestra.run.smithi032.stdout:5212 write 0x882f0b9 thru 0x8839324 (0xa26c bytes) 2024-09-13T09:18:53.828 INFO:teuthology.orchestra.run.smithi032.stdout:5213 writesame 0xe9372e thru 0xe9cb27 data_size 0x3e(0x93fa bytes) 2024-09-13T09:18:53.835 INFO:teuthology.orchestra.run.smithi032.stdout:5215 trunc from 0x8839325 to 0xb976371 2024-09-13T09:18:53.843 INFO:teuthology.orchestra.run.smithi032.stdout:5217 read 0x553458 thru 0x55dfe2 (0xab8b bytes) 2024-09-13T09:18:53.843 INFO:teuthology.orchestra.run.smithi032.stdout:5218 read 0x15f3613 thru 0x15f771e (0x410c bytes) 2024-09-13T09:18:53.844 INFO:teuthology.orchestra.run.smithi032.stdout:5219 write 0x93583db thru 0x935b431 (0x3057 bytes) 2024-09-13T09:18:53.846 INFO:teuthology.orchestra.run.smithi032.stdout:5220 write 0x28de525 thru 0x28e1994 (0x3470 bytes) 2024-09-13T09:18:53.848 INFO:teuthology.orchestra.run.smithi032.stdout:5221 writesame 0x9fe33cf thru 0x9fe374d data_size 0x1(0x37f bytes) 2024-09-13T09:18:53.851 INFO:teuthology.orchestra.run.smithi032.stdout:5222 writesame 0x6e07453 thru 0x6e16c71 data_size 0x1f(0xf81f bytes) 2024-09-13T09:18:53.854 INFO:teuthology.orchestra.run.smithi032.stdout:5223 clone 39 order 23 su 4194304 sc 9 2024-09-13T09:18:53.872 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:53.873 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:53.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:53.895 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:53.893+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:53.996 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:53.990+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:54.015 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:54.016 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:54.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:54.016 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:54.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.016 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:18:54.016 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:54.016 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.326787761000006, 2024-09-13T09:18:54.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.21372158699999999, 2024-09-13T09:18:54.016 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.016 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:54.016 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.016 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.017 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10 2024-09-13T09:18:54.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.017 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:54.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:54.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685363+0000", 2024-09-13T09:18:54.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:54.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685372+0000", 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4590000000000004e-06 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685394+0000", 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1616000000000001e-05 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685407+0000", 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3675e-05 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685484+0000", 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7065999999999997e-05 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685493+0000", 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8070000000000006e-06 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690770+0000", 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052763869999999996 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690940+0000", 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017001900000000001 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.691238+0000", 2024-09-13T09:18:54.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029794499999999998 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.692057+0000", 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081943000000000001 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.898933+0000", 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.20687588200000001 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899026+0000", 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2777999999999998e-05 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899086+0000", 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0188e-05 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.023 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:14590 3.1d 3:be60f898:::rbd_data.11853ecd5f5f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13672152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 66.470939321000003, 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052557382, 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 14590 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541207+0000", 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-13T09:18:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562477+0000", 2024-09-13T09:18:54.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021270155999999998 2024-09-13T09:18:54.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562539+0000", 2024-09-13T09:18:54.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1851000000000001e-05 2024-09-13T09:18:54.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562565+0000", 2024-09-13T09:18:54.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6194e-05 2024-09-13T09:18:54.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562702+0000", 2024-09-13T09:18:54.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013731299999999999 2024-09-13T09:18:54.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.563071+0000", 2024-09-13T09:18:54.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036892899999999999 2024-09-13T09:18:54.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:54.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.565167+0000", 2024-09-13T09:18:54.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020961959999999998 2024-09-13T09:18:54.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:54.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.585261+0000", 2024-09-13T09:18:54.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020093443999999999 2024-09-13T09:18:54.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:54.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593596+0000", 2024-09-13T09:18:54.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083352849999999996 2024-09-13T09:18:54.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:54.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593696+0000", 2024-09-13T09:18:54.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100021 2024-09-13T09:18:54.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593770+0000", 2024-09-13T09:18:54.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3889999999999999e-05 2024-09-13T09:18:54.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.033 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.033 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:14584 3.2 e39/22 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head v 39'64, mlcod=38'62)", 2024-09-13T09:18:54.033 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:54.034 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 66.468953674999995, 2024-09-13T09:18:54.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051546091000000002, 2024-09-13T09:18:54.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:54.034 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:54.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:54.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543197+0000", 2024-09-13T09:18:54.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:54.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588691+0000", 2024-09-13T09:18:54.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045493475999999998 2024-09-13T09:18:54.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588711+0000", 2024-09-13T09:18:54.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0485000000000001e-05 2024-09-13T09:18:54.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588721+0000", 2024-09-13T09:18:54.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.499e-06 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588766+0000", 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5228000000000001e-05 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588788+0000", 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2503000000000001e-05 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:54.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594671+0000", 2024-09-13T09:18:54.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058822809999999996 2024-09-13T09:18:54.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594744+0000", 2024-09-13T09:18:54.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3776000000000007e-05 2024-09-13T09:18:54.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.040 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.041 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:22580 3.1c 3:39876c9f:::rbd_data.1185d6f8ec87.000000000000000f:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T09:18:54.041 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:54.041 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.159700884999999, 2024-09-13T09:18:54.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050372896, 2024-09-13T09:18:54.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.041 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:54.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22580 2024-09-13T09:18:54.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.041 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852449+0000", 2024-09-13T09:18:54.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:54.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852462+0000", 2024-09-13T09:18:54.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3756e-05 2024-09-13T09:18:54.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852495+0000", 2024-09-13T09:18:54.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2435999999999999e-05 2024-09-13T09:18:54.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852516+0000", 2024-09-13T09:18:54.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1347000000000001e-05 2024-09-13T09:18:54.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852644+0000", 2024-09-13T09:18:54.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012777200000000001 2024-09-13T09:18:54.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852784+0000", 2024-09-13T09:18:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013980500000000001 2024-09-13T09:18:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.860936+0000", 2024-09-13T09:18:54.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081526859999999993 2024-09-13T09:18:54.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:54.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902522+0000", 2024-09-13T09:18:54.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041585203000000001 2024-09-13T09:18:54.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:54.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902560+0000", 2024-09-13T09:18:54.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8603000000000003e-05 2024-09-13T09:18:54.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:54.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902708+0000", 2024-09-13T09:18:54.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014740900000000001 2024-09-13T09:18:54.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902824+0000", 2024-09-13T09:18:54.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116519 2024-09-13T09:18:54.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.052 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.052 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54746 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8011776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.052 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:54.052 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.780770408999999, 2024-09-13T09:18:54.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041426971999999999, 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54746 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231381+0000", 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231384+0000", 2024-09-13T09:18:54.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.599e-06 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231390+0000", 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231395+0000", 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0170000000000004e-06 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272716+0000", 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041321670999999997 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272767+0000", 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0447e-05 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272809+0000", 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1659999999999998e-05 2024-09-13T09:18:54.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.057 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.057 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54986 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6348800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.057 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:54.057 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769861809999998, 2024-09-13T09:18:54.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041375935000000003, 2024-09-13T09:18:54.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54986 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242292+0000", 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3039999999999999e-06 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242298+0000", 2024-09-13T09:18:54.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242303+0000", 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8359999999999999e-06 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283539+0000", 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041235330000000001 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283593+0000", 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4926000000000002e-05 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283666+0000", 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2718000000000004e-05 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.061 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54987 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6447104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769844628000001, 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041604369000000002, 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54987 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242309+0000", 2024-09-13T09:18:54.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1160000000000002e-06 2024-09-13T09:18:54.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242317+0000", 2024-09-13T09:18:54.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9799999999999998e-06 2024-09-13T09:18:54.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242323+0000", 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6439999999999997e-06 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283711+0000", 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388085999999998 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283766+0000", 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5112999999999999e-05 2024-09-13T09:18:54.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283912+0000", 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014575300000000001 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54988 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6545408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769824807999999, 2024-09-13T09:18:54.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041865404000000002, 2024-09-13T09:18:54.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54988 2024-09-13T09:18:54.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.067 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:54.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:54.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:54.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242329+0000", 2024-09-13T09:18:54.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:18:54.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242335+0000", 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242341+0000", 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9440000000000002e-06 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283952+0000", 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041610981999999998 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284076+0000", 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124356 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284193+0000", 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116345 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.071 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.072 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.072 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54989 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6643712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.072 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:54.072 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769806694, 2024-09-13T09:18:54.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042123657000000002, 2024-09-13T09:18:54.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54989 2024-09-13T09:18:54.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.073 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:54.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242348+0000", 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.655e-06 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242354+0000", 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2269999999999998e-06 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242360+0000", 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6170000000000003e-06 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284248+0000", 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041887830000000001 2024-09-13T09:18:54.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284340+0000", 2024-09-13T09:18:54.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1358000000000002e-05 2024-09-13T09:18:54.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284469+0000", 2024-09-13T09:18:54.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012932300000000001 2024-09-13T09:18:54.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.076 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.077 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54990 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6742016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.077 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:54.077 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769773403999999, 2024-09-13T09:18:54.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042278333000000001, 2024-09-13T09:18:54.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54990 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:54.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242378+0000", 2024-09-13T09:18:54.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242381+0000", 2024-09-13T09:18:54.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0680000000000001e-06 2024-09-13T09:18:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242392+0000", 2024-09-13T09:18:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0762e-05 2024-09-13T09:18:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242398+0000", 2024-09-13T09:18:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1040000000000003e-06 2024-09-13T09:18:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284506+0000", 2024-09-13T09:18:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042107983000000002 2024-09-13T09:18:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284564+0000", 2024-09-13T09:18:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.821e-05 2024-09-13T09:18:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284657+0000", 2024-09-13T09:18:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2588000000000002e-05 2024-09-13T09:18:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54991 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6840320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769748613000001, 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042489459, 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54991 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242406+0000", 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242412+0000", 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5919999999999997e-06 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242425+0000", 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2713e-05 2024-09-13T09:18:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284694+0000", 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042269558999999998 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284784+0000", 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9999000000000003e-05 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284893+0000", 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108422 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54992 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6938624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769722242, 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042636897999999999, 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54992 2024-09-13T09:18:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242429+0000", 2024-09-13T09:18:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242432+0000", 2024-09-13T09:18:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7599999999999998e-06 2024-09-13T09:18:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242439+0000", 2024-09-13T09:18:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5760000000000002e-06 2024-09-13T09:18:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242448+0000", 2024-09-13T09:18:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-13T09:18:54.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284935+0000", 2024-09-13T09:18:54.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042486949000000003 2024-09-13T09:18:54.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284991+0000", 2024-09-13T09:18:54.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6471000000000002e-05 2024-09-13T09:18:54.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285067+0000", 2024-09-13T09:18:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5543000000000005e-05 2024-09-13T09:18:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54993 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7036928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769699365000001, 2024-09-13T09:18:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042867959999999997, 2024-09-13T09:18:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54993 2024-09-13T09:18:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242452+0000", 2024-09-13T09:18:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242455+0000", 2024-09-13T09:18:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4360000000000001e-06 2024-09-13T09:18:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242463+0000", 2024-09-13T09:18:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8220000000000007e-06 2024-09-13T09:18:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242468+0000", 2024-09-13T09:18:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9479999999999999e-06 2024-09-13T09:18:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285107+0000", 2024-09-13T09:18:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042639415999999999 2024-09-13T09:18:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285196+0000", 2024-09-13T09:18:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8721000000000007e-05 2024-09-13T09:18:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285321+0000", 2024-09-13T09:18:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012495 2024-09-13T09:18:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54994 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7135232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769672200999999, 2024-09-13T09:18:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043009622999999997, 2024-09-13T09:18:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54994 2024-09-13T09:18:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242482+0000", 2024-09-13T09:18:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:18:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242486+0000", 2024-09-13T09:18:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1049999999999997e-06 2024-09-13T09:18:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242489+0000", 2024-09-13T09:18:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.004e-06 2024-09-13T09:18:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285354+0000", 2024-09-13T09:18:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.04286479 2024-09-13T09:18:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285413+0000", 2024-09-13T09:18:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9778000000000003e-05 2024-09-13T09:18:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285489+0000", 2024-09-13T09:18:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6141000000000002e-05 2024-09-13T09:18:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54995 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7233536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769660829999999, 2024-09-13T09:18:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043166641999999998, 2024-09-13T09:18:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54995 2024-09-13T09:18:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242492+0000", 2024-09-13T09:18:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.437e-06 2024-09-13T09:18:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242496+0000", 2024-09-13T09:18:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:18:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242498+0000", 2024-09-13T09:18:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5950000000000001e-06 2024-09-13T09:18:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285528+0000", 2024-09-13T09:18:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043029755000000003 2024-09-13T09:18:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285585+0000", 2024-09-13T09:18:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:18:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285658+0000", 2024-09-13T09:18:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2713999999999995e-05 2024-09-13T09:18:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54996 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7331840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769651515, 2024-09-13T09:18:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043333782000000001, 2024-09-13T09:18:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54996 2024-09-13T09:18:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242500+0000", 2024-09-13T09:18:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242502+0000", 2024-09-13T09:18:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3599999999999999e-06 2024-09-13T09:18:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242505+0000", 2024-09-13T09:18:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1439999999999999e-06 2024-09-13T09:18:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242507+0000", 2024-09-13T09:18:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.605e-06 2024-09-13T09:18:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285698+0000", 2024-09-13T09:18:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043190911999999998 2024-09-13T09:18:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285754+0000", 2024-09-13T09:18:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5541999999999998e-05 2024-09-13T09:18:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285834+0000", 2024-09-13T09:18:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0426000000000006e-05 2024-09-13T09:18:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54997 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769642426000001, 2024-09-13T09:18:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043496174999999998, 2024-09-13T09:18:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54997 2024-09-13T09:18:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242509+0000", 2024-09-13T09:18:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242511+0000", 2024-09-13T09:18:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.313e-06 2024-09-13T09:18:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242514+0000", 2024-09-13T09:18:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:18:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242516+0000", 2024-09-13T09:18:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3999999999999999e-06 2024-09-13T09:18:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285875+0000", 2024-09-13T09:18:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043358578000000002 2024-09-13T09:18:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285929+0000", 2024-09-13T09:18:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4602999999999999e-05 2024-09-13T09:18:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286006+0000", 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6429999999999998e-05 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54998 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7528448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.769549814000001, 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043666469999999999, 2024-09-13T09:18:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54998 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242518+0000", 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999161 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242606+0000", 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7607000000000003e-05 2024-09-13T09:18:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242617+0000", 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0882e-05 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242624+0000", 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9009999999999997e-06 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286046+0000", 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043422308 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286106+0000", 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0285000000000003e-05 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286269+0000", 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000162578 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:55006 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8314880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.766161146000002, 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041461771000000001, 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 55006 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245993+0000", 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2869999999999999e-06 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245999+0000", 2024-09-13T09:18:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0159999999999997e-06 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.246004+0000", 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287345+0000", 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041341217 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287398+0000", 2024-09-13T09:18:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2840000000000002e-05 2024-09-13T09:18:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287453+0000", 2024-09-13T09:18:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4746000000000001e-05 2024-09-13T09:18:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:68662 3.1d 3:be8be2fa:::rbd_data.118556027ff4.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.855105705, 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.044923689000000003, 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 68662 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157045+0000", 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157052+0000", 2024-09-13T09:18:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8750000000000002e-06 2024-09-13T09:18:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157066+0000", 2024-09-13T09:18:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3815000000000001e-05 2024-09-13T09:18:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157073+0000", 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.64e-06 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157145+0000", 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2517999999999999e-05 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157267+0000", 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012185599999999999 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.164012+0000", 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0067447660000000001 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201805+0000", 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037793528999999999 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201881+0000", 2024-09-13T09:18:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5678000000000003e-05 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201933+0000", 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2346999999999999e-05 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201970+0000", 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6532000000000002e-05 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.159 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:54.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:54.161 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:54.157+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:54.261 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:54.257+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:54.335 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:54.335 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:54.349 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:54.362 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:54.361+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:54.463 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:54.461+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:54.468 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:54.469 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:54.469 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:54.469 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:54.469 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:54.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:54.564 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:54.561+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:54.575 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-13T09:18:54.611 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:50082 3.10 3:0ec43ecb:::rbd_object_map.1185172eb410:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T09:18:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.245850520000001, 2024-09-13T09:18:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80943783999999996, 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 50082 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362619+0000", 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362632+0000", 2024-09-13T09:18:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3338e-05 2024-09-13T09:18:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362657+0000", 2024-09-13T09:18:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5207000000000001e-05 2024-09-13T09:18:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362669+0000", 2024-09-13T09:18:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1768e-05 2024-09-13T09:18:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362738+0000", 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9486999999999994e-05 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362748+0000", 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0010000000000001e-05 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169473+0000", 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80672482300000004 2024-09-13T09:18:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169884+0000", 2024-09-13T09:18:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041114299999999999 2024-09-13T09:18:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.170206+0000", 2024-09-13T09:18:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032127700000000002 2024-09-13T09:18:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171827+0000", 2024-09-13T09:18:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016217880000000001 2024-09-13T09:18:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171871+0000", 2024-09-13T09:18:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:18:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171967+0000", 2024-09-13T09:18:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5752000000000001e-05 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.172059+0000", 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2082000000000004e-05 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56772 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 4112384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.178162401000002, 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054477114, 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56772 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430309+0000", 2024-09-13T09:18:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430312+0000", 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.255e-06 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430319+0000", 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4760000000000001e-06 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430324+0000", 2024-09-13T09:18:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9799999999999998e-06 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484634+0000", 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054310031000000002 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484697+0000", 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3355999999999994e-05 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484787+0000", 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9350000000000003e-05 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56785 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5390336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174378918999999, 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054655007999999998, 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56785 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434094+0000", 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.471e-06 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434097+0000", 2024-09-13T09:18:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0740000000000001e-06 2024-09-13T09:18:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434100+0000", 2024-09-13T09:18:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7829999999999999e-06 2024-09-13T09:18:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488422+0000", 2024-09-13T09:18:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054321855000000002 2024-09-13T09:18:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488563+0000", 2024-09-13T09:18:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014154 2024-09-13T09:18:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488748+0000", 2024-09-13T09:18:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018461399999999999 2024-09-13T09:18:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56786 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174369537, 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054950539999999999, 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56786 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434104+0000", 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4020000000000001e-06 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434107+0000", 2024-09-13T09:18:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.072e-06 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434109+0000", 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.717e-06 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488795+0000", 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054685661000000003 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488869+0000", 2024-09-13T09:18:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4265999999999997e-05 2024-09-13T09:18:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489053+0000", 2024-09-13T09:18:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018374299999999999 2024-09-13T09:18:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56787 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5586944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174359278000001, 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055115725999999997, 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56787 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434112+0000", 2024-09-13T09:18:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434114+0000", 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.308e-06 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434117+0000", 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434119+0000", 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.588e-06 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489097+0000", 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054977824000000002 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489146+0000", 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8503e-05 2024-09-13T09:18:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489228+0000", 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2694000000000005e-05 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56788 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174350387, 2024-09-13T09:18:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055437562000000003, 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56788 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434121+0000", 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434123+0000", 2024-09-13T09:18:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2190000000000001e-06 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434126+0000", 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434128+0000", 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8250000000000001e-06 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489293+0000", 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055164092999999997 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489428+0000", 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013572599999999999 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489559+0000", 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013086 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56789 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174341238, 2024-09-13T09:18:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055674077000000002, 2024-09-13T09:18:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56789 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434132+0000", 2024-09-13T09:18:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.302e-06 2024-09-13T09:18:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434136+0000", 2024-09-13T09:18:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-13T09:18:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434139+0000", 2024-09-13T09:18:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:18:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489583+0000", 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055443994000000003 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489699+0000", 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116611 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489805+0000", 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105331 2024-09-13T09:18:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56790 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174330863000002, 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055716839999999997, 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56790 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434142+0000", 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2559999999999999e-06 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434145+0000", 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.96e-06 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434148+0000", 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6479999999999999e-06 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489851+0000", 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055702792000000001 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489858+0000", 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3900000000000004e-06 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56791 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174322091000001, 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055740029000000003, 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56791 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434151+0000", 2024-09-13T09:18:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434154+0000", 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0139999999999999e-06 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434157+0000", 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6699999999999998e-06 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489884+0000", 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055727511 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489890+0000", 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7200000000000003e-06 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56792 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174313162000001, 2024-09-13T09:18:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055760363, 2024-09-13T09:18:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56792 2024-09-13T09:18:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434163+0000", 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434167+0000", 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.292e-06 2024-09-13T09:18:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434169+0000", 2024-09-13T09:18:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:18:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489914+0000", 2024-09-13T09:18:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055744415999999998 2024-09-13T09:18:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489919+0000", 2024-09-13T09:18:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:18:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56793 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174299642000001, 2024-09-13T09:18:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055774049999999999, 2024-09-13T09:18:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56793 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434173+0000", 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-06 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434177+0000", 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1070000000000001e-06 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434179+0000", 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.666e-06 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489941+0000", 2024-09-13T09:18:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055762071000000003 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489946+0000", 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56794 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174290534000001, 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055791704999999997, 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56794 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434182+0000", 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1629999999999999e-06 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434185+0000", 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9950000000000001e-06 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434188+0000", 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489968+0000", 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055779789000000003 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489973+0000", 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2800000000000003e-06 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56795 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174281598, 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055809612000000001, 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56795 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434191+0000", 2024-09-13T09:18:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.212e-06 2024-09-13T09:18:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434194+0000", 2024-09-13T09:18:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.078e-06 2024-09-13T09:18:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434198+0000", 2024-09-13T09:18:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9820000000000002e-06 2024-09-13T09:18:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489995+0000", 2024-09-13T09:18:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055796363000000002 2024-09-13T09:18:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490000+0000", 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1839999999999998e-06 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56796 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174271337, 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055825859999999998, 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56796 2024-09-13T09:18:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434200+0000", 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434202+0000", 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.204e-06 2024-09-13T09:18:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434205+0000", 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9689999999999999e-06 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434207+0000", 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490021+0000", 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055813929999999998 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490027+0000", 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.259e-06 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56797 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.174262618, 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055843649000000002, 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56797 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:54.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434210+0000", 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2389999999999999e-06 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434213+0000", 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434216+0000", 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7800000000000001e-06 2024-09-13T09:18:54.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490048+0000", 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055831981000000003 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490053+0000", 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8640000000000004e-06 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56807 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.172582460000001, 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054407720999999999, 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56807 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435889+0000", 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435891+0000", 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.217e-06 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435894+0000", 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.067e-06 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435896+0000", 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490292+0000", 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054395840000000001 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490297+0000", 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.685 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.686 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.686 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56808 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.686 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:54.686 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.172573633999999, 2024-09-13T09:18:54.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05442524, 2024-09-13T09:18:54.686 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.686 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.686 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.686 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.686 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.687 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56808 2024-09-13T09:18:54.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.687 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:54.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:54.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435900+0000", 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435903+0000", 2024-09-13T09:18:54.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:18:54.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435905+0000", 2024-09-13T09:18:54.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:18:54.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490319+0000", 2024-09-13T09:18:54.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054413396000000003 2024-09-13T09:18:54.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490324+0000", 2024-09-13T09:18:54.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:18:54.690 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.690 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.690 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56809 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.172563577999998, 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054442786, 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56809 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:54.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:54.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435910+0000", 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435913+0000", 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1470000000000001e-06 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435915+0000", 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6769999999999999e-06 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490346+0000", 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054430574000000002 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490351+0000", 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2700000000000004e-06 2024-09-13T09:18:54.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56810 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.172554279, 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054459855000000001, 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56810 2024-09-13T09:18:54.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.696 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:54.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:54.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435917+0000", 2024-09-13T09:18:54.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435919+0000", 2024-09-13T09:18:54.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.457e-06 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435924+0000", 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435929+0000", 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8300000000000003e-06 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490373+0000", 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054443841999999999 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490378+0000", 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8509999999999998e-06 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56811 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.172525178000001, 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054456683999999998, 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56811 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:54.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:54.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:54.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:54.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:54.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:54.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435946+0000", 2024-09-13T09:18:54.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:54.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435949+0000", 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435956+0000", 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8719999999999996e-06 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:54.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435962+0000", 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000004e-06 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490398+0000", 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054436824000000002 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490403+0000", 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0710000000000002e-06 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.703 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:54.704 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:54.704 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:54.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:54.706 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:54.702+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:54.795 INFO:teuthology.orchestra.run.smithi032.stdout:truncating image image_client.0-clone38 from 0xb976371 (overlap 0x95f799) to 0x0 2024-09-13T09:18:54.807 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:54.802+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:54.889 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:54.890 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:54.904 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:54.946 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:54.942+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:55.008 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:55.005+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:55.076 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:55.077 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:55.094 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:55.101 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-13T09:18:55.138 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:55.134+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:55.176 INFO:teuthology.orchestra.run.smithi032.stdout:5224 read 0x4f33d5e thru 0x4f36656 (0x28f9 bytes) 2024-09-13T09:18:55.179 INFO:teuthology.orchestra.run.smithi032.stdout:5225 punch from 0x2e22251 to 0x2e2e848, (0xc5f7 bytes) 2024-09-13T09:18:55.203 INFO:teuthology.orchestra.run.smithi032.stdout:5226 trunc from 0xb976371 to 0x5a616dd 2024-09-13T09:18:55.210 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:55.205+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:55.221 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:18:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:18:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.826764238, 2024-09-13T09:18:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077478195999999999, 2024-09-13T09:18:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392636+0000", 2024-09-13T09:18:55.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:55.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392651+0000", 2024-09-13T09:18:55.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:55.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392674+0000", 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.264e-05 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392691+0000", 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7203000000000001e-05 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392780+0000", 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8671999999999994e-05 2024-09-13T09:18:55.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392794+0000", 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.466664+0000", 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073869913999999995 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467139+0000", 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047488599999999999 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467664+0000", 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052530600000000002 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469125+0000", 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001460432 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469847+0000", 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072197799999999996 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470006+0000", 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015955900000000001 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470116+0000", 2024-09-13T09:18:55.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109836 2024-09-13T09:18:55.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.229 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.229 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:49389 3.d 3:b0b9272f:::rbd_data.1185e728d948.0000000000000025:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-13T09:18:55.229 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:18:55.229 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.136445462000001, 2024-09-13T09:18:55.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077194925999999997, 2024-09-13T09:18:55.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 49389 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082955+0000", 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082967+0000", 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2609000000000001e-05 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082998+0000", 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0596000000000003e-05 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083013+0000", 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4766999999999999e-05 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083136+0000", 2024-09-13T09:18:55.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012295599999999999 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083360+0000", 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000224661 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.097720+0000", 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014359627 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.159975+0000", 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062254728000000002 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.159999+0000", 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.427e-05 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.160089+0000", 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9895999999999994e-05 2024-09-13T09:18:55.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.160152+0000", 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2939000000000002e-05 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52425 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6111232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805609+0000", 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.413792837999999, 2024-09-13T09:18:55.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064134332000000002, 2024-09-13T09:18:55.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52425 2024-09-13T09:18:55.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.236 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805609+0000", 2024-09-13T09:18:55.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805609+0000", 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805609+0000", 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805612+0000", 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.27e-06 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805620+0000", 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805627+0000", 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8669999999999996e-06 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869445+0000", 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063818099000000003 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869583+0000", 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138078 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869744+0000", 2024-09-13T09:18:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160462 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52426 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6209536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.413769181999999, 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064408950000000006, 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52426 2024-09-13T09:18:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:18:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805632+0000", 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805645+0000", 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2469999999999999e-05 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805654+0000", 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7760000000000003e-06 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805661+0000", 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.114e-06 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869801+0000", 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064139777999999995 2024-09-13T09:18:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869918+0000", 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117199 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870042+0000", 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124196 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52427 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6307840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.413735251999999, 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064720860000000005, 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52427 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805666+0000", 2024-09-13T09:18:55.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805670+0000", 2024-09-13T09:18:55.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2200000000000001e-06 2024-09-13T09:18:55.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805678+0000", 2024-09-13T09:18:55.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.32e-06 2024-09-13T09:18:55.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805685+0000", 2024-09-13T09:18:55.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.883e-06 2024-09-13T09:18:55.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870103+0000", 2024-09-13T09:18:55.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064417962999999995 2024-09-13T09:18:55.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870241+0000", 2024-09-13T09:18:55.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138362 2024-09-13T09:18:55.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870388+0000", 2024-09-13T09:18:55.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146732 2024-09-13T09:18:55.250 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.250 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.250 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.250 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52428 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.250 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:18:55.250 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.413709522000001, 2024-09-13T09:18:55.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064990774000000001, 2024-09-13T09:18:55.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.250 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52428 2024-09-13T09:18:55.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.251 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:18:55.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:18:55.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805692+0000", 2024-09-13T09:18:55.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805695+0000", 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805704+0000", 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5299999999999996e-06 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805712+0000", 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870433+0000", 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064720343999999999 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870544+0000", 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011108900000000001 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870683+0000", 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000139705 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52429 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6504448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.413669226, 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065238731999999994, 2024-09-13T09:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52429 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805732+0000", 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805736+0000", 2024-09-13T09:18:55.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8600000000000003e-06 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805746+0000", 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6860000000000001e-06 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805753+0000", 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870724+0000", 2024-09-13T09:18:55.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064970450999999999 2024-09-13T09:18:55.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870836+0000", 2024-09-13T09:18:55.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112796 2024-09-13T09:18:55.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870972+0000", 2024-09-13T09:18:55.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135382 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52430 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6602752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.413643, 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065499180000000004, 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52430 2024-09-13T09:18:55.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.261 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:18:55.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805762+0000", 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805773+0000", 2024-09-13T09:18:55.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1496e-05 2024-09-13T09:18:55.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805780+0000", 2024-09-13T09:18:55.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:18:55.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871018+0000", 2024-09-13T09:18:55.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065237396000000003 2024-09-13T09:18:55.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871124+0000", 2024-09-13T09:18:55.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106469 2024-09-13T09:18:55.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871258+0000", 2024-09-13T09:18:55.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134175 2024-09-13T09:18:55.264 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.265 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.265 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52431 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6701056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.265 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:18:55.265 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.413616145999999, 2024-09-13T09:18:55.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065755623999999999, 2024-09-13T09:18:55.265 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.265 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52431 2024-09-13T09:18:55.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.266 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:18:55.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:18:55.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805785+0000", 2024-09-13T09:18:55.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805789+0000", 2024-09-13T09:18:55.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2160000000000002e-06 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805797+0000", 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4870000000000002e-06 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805804+0000", 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871298+0000", 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065494259999999999 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871405+0000", 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106512 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871542+0000", 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013683599999999999 2024-09-13T09:18:55.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52432 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6799360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.413551447, 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065976444999999995, 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52432 2024-09-13T09:18:55.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.271 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:18:55.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:18:55.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805850+0000", 2024-09-13T09:18:55.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:55.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805855+0000", 2024-09-13T09:18:55.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8670000000000002e-06 2024-09-13T09:18:55.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805867+0000", 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2597e-05 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805876+0000", 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.833e-06 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871584+0000", 2024-09-13T09:18:55.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065707765000000001 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871690+0000", 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105832 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871827+0000", 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013744500000000001 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.274 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52440 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7585792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.409427634, 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064276712999999999, 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52440 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:18:55.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:18:55.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809974+0000", 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809976+0000", 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8920000000000001e-06 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809981+0000", 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5170000000000001e-06 2024-09-13T09:18:55.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809985+0000", 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9539999999999998e-06 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.873936+0000", 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063951470999999996 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874076+0000", 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014022199999999999 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874251+0000", 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017497500000000001 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52441 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7684096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:18:55.280 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.409414255000002, 2024-09-13T09:18:55.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064567507999999996, 2024-09-13T09:18:55.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52441 2024-09-13T09:18:55.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.280 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809989+0000", 2024-09-13T09:18:55.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8589999999999999e-06 2024-09-13T09:18:55.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809994+0000", 2024-09-13T09:18:55.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:18:55.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809999+0000", 2024-09-13T09:18:55.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:18:55.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874317+0000", 2024-09-13T09:18:55.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064318742999999998 2024-09-13T09:18:55.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874423+0000", 2024-09-13T09:18:55.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106116 2024-09-13T09:18:55.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874555+0000", 2024-09-13T09:18:55.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013196700000000001 2024-09-13T09:18:55.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.284 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.285 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52442 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7782400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.285 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:18:55.285 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.409400252000001, 2024-09-13T09:18:55.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064804082999999998, 2024-09-13T09:18:55.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.285 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.285 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.285 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.285 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52442 2024-09-13T09:18:55.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.286 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:18:55.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:18:55.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810003+0000", 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8360000000000001e-06 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810009+0000", 2024-09-13T09:18:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0220000000000001e-06 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810013+0000", 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9879999999999998e-06 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874593+0000", 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064579851999999993 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874688+0000", 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4581999999999997e-05 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874806+0000", 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118108 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.289 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52443 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7880704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.409385356000001, 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065175650000000002, 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52443 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810018+0000", 2024-09-13T09:18:55.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.798e-06 2024-09-13T09:18:55.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810023+0000", 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7829999999999998e-06 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810027+0000", 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8890000000000004e-06 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874886+0000", 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064859362000000004 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875029+0000", 2024-09-13T09:18:55.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014311900000000001 2024-09-13T09:18:55.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875192+0000", 2024-09-13T09:18:55.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000163012 2024-09-13T09:18:55.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.294 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.295 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52444 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7979008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.295 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:18:55.295 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.409371777, 2024-09-13T09:18:55.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065471830999999994, 2024-09-13T09:18:55.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.295 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52444 2024-09-13T09:18:55.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.296 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:18:55.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:18:55.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:18:55.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810032+0000", 2024-09-13T09:18:55.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8199999999999999e-06 2024-09-13T09:18:55.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810036+0000", 2024-09-13T09:18:55.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5510000000000001e-06 2024-09-13T09:18:55.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810041+0000", 2024-09-13T09:18:55.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0679999999999998e-06 2024-09-13T09:18:55.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875256+0000", 2024-09-13T09:18:55.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065215765999999994 2024-09-13T09:18:55.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875366+0000", 2024-09-13T09:18:55.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109639 2024-09-13T09:18:55.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875502+0000", 2024-09-13T09:18:55.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013630399999999999 2024-09-13T09:18:55.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.300 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.300 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52445 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:18:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.409358461, 2024-09-13T09:18:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065733503999999998, 2024-09-13T09:18:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52445 2024-09-13T09:18:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.301 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:18:55.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:18:55.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810043+0000", 2024-09-13T09:18:55.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810045+0000", 2024-09-13T09:18:55.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.813e-06 2024-09-13T09:18:55.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810050+0000", 2024-09-13T09:18:55.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5940000000000004e-06 2024-09-13T09:18:55.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810054+0000", 2024-09-13T09:18:55.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439999999999998e-06 2024-09-13T09:18:55.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875543+0000", 2024-09-13T09:18:55.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065489393000000007 2024-09-13T09:18:55.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875658+0000", 2024-09-13T09:18:55.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115302 2024-09-13T09:18:55.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875777+0000", 2024-09-13T09:18:55.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118776 2024-09-13T09:18:55.306 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.306 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.306 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.306 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52446 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8175616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.306 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:18:55.306 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.409345131999999, 2024-09-13T09:18:55.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065969204000000004, 2024-09-13T09:18:55.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.307 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.307 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.307 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.307 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52446 2024-09-13T09:18:55.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.307 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:18:55.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:18:55.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:18:55.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810059+0000", 2024-09-13T09:18:55.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.84e-06 2024-09-13T09:18:55.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810063+0000", 2024-09-13T09:18:55.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4329999999999997e-06 2024-09-13T09:18:55.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810067+0000", 2024-09-13T09:18:55.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8800000000000001e-06 2024-09-13T09:18:55.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875813+0000", 2024-09-13T09:18:55.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065745910000000005 2024-09-13T09:18:55.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875903+0000", 2024-09-13T09:18:55.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0665e-05 2024-09-13T09:18:55.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876026+0000", 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122836 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52447 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8273920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.409310891000001, 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066176867, 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.312 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52447 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:18:55.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:18:55.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810093+0000", 2024-09-13T09:18:55.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6089999999999999e-06 2024-09-13T09:18:55.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810099+0000", 2024-09-13T09:18:55.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1539999999999999e-06 2024-09-13T09:18:55.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810104+0000", 2024-09-13T09:18:55.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4630000000000003e-06 2024-09-13T09:18:55.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876062+0000", 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065957672999999994 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876153+0000", 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1119000000000006e-05 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876268+0000", 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115431 2024-09-13T09:18:55.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52448 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.409294722999999, 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066534826000000005, 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52448 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810109+0000", 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0169999999999999e-06 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810115+0000", 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:18:55.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810120+0000", 2024-09-13T09:18:55.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9620000000000001e-06 2024-09-13T09:18:55.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876349+0000", 2024-09-13T09:18:55.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066229213999999995 2024-09-13T09:18:55.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876484+0000", 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013503199999999999 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876642+0000", 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158228 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.321 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52452 3.d 3:b5f40919:::rbd_data.118567e41306.0000000000000055:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.409278884999999, 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070307575999999997, 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52452 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810126+0000", 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810132+0000", 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7349999999999999e-06 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:55.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810136+0000", 2024-09-13T09:18:55.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2069999999999997e-06 2024-09-13T09:18:55.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:55.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876306+0000", 2024-09-13T09:18:55.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066169943999999994 2024-09-13T09:18:55.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:55.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876520+0000", 2024-09-13T09:18:55.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021377500000000001 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.877399+0000", 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087836900000000005 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880158+0000", 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002759657 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880212+0000", 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3551999999999998e-05 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880315+0000", 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010306099999999999 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880431+0000", 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011603300000000001 2024-09-13T09:18:55.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.328 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.328 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.328 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:55.328 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:55.328 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:55.330 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:55.326+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:55.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:55.331 INFO:teuthology.orchestra.run.smithi032.stdout:5227 read 0x23a9ce1 thru 0x23b90a9 (0xf3c9 bytes) 2024-09-13T09:18:55.331 INFO:teuthology.orchestra.run.smithi032.stdout:5228 trunc from 0x5a616dd to 0x5c41eed 2024-09-13T09:18:55.331 INFO:teuthology.orchestra.run.smithi032.stdout:5229 punch from 0x1ab9787 to 0x1ac1951, (0x81ca bytes) 2024-09-13T09:18:55.332 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:55.332 INFO:teuthology.orchestra.run.smithi032.stdout:5231 write 0xf4f90a thru 0xf5f6e3 (0xfdda bytes) 2024-09-13T09:18:55.332 INFO:teuthology.orchestra.run.smithi032.stdout:5232 write 0xb1478c thru 0xb23809 (0xf07e bytes) 2024-09-13T09:18:55.332 INFO:teuthology.orchestra.run.smithi032.stdout:5234 read 0x401d114 thru 0x402c430 (0xf31d bytes) 2024-09-13T09:18:55.332 INFO:teuthology.orchestra.run.smithi032.stdout:5235 trunc from 0x5c41eed to 0x6a44f76 2024-09-13T09:18:55.332 INFO:teuthology.orchestra.run.smithi032.stdout:5237 trunc from 0x6a44f76 to 0x49fb962 2024-09-13T09:18:55.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:18:55.355 INFO:teuthology.orchestra.run.smithi032.stdout:5238 write 0x794b592 thru 0x7957f76 (0xc9e5 bytes) 2024-09-13T09:18:55.389 INFO:teuthology.orchestra.run.smithi032.stdout:5239 write 0xeaed9b3 thru 0xeaf3177 (0x57c5 bytes) 2024-09-13T09:18:55.395 INFO:teuthology.orchestra.run.smithi032.stdout:5240 write 0xfa01da thru 0xfa7b38 (0x795f bytes) 2024-09-13T09:18:55.402 INFO:teuthology.orchestra.run.smithi032.stdout:5241 trunc from 0xeaf3178 to 0xccdc093 2024-09-13T09:18:55.422 INFO:teuthology.orchestra.run.smithi032.stdout:5242 punch from 0x3765490 to 0x3772536, (0xd0a6 bytes) 2024-09-13T09:18:55.423 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:55.423 INFO:teuthology.orchestra.run.smithi032.stdout:5246 writesame 0x1196781 thru 0x1198d15 data_size 0x3(0x2595 bytes) 2024-09-13T09:18:55.430 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:55.426+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:55.436 INFO:teuthology.orchestra.run.smithi032.stdout:5248 read 0x79c9fb4 thru 0x79d16b9 (0x7706 bytes) 2024-09-13T09:18:55.437 INFO:teuthology.orchestra.run.smithi032.stdout:5250 writesame 0x1571453 thru 0x1573845 data_size 0x1(0x23f3 bytes) 2024-09-13T09:18:55.449 INFO:teuthology.orchestra.run.smithi032.stdout:5251 write 0x12a760 thru 0x13839d (0xdc3e bytes) 2024-09-13T09:18:55.481 INFO:teuthology.orchestra.run.smithi032.stdout:5254 writesame 0xb9ce224 thru 0xb9dd01d data_size 0x2(0xedfa bytes) 2024-09-13T09:18:55.487 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:55.487 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:55.487 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:55.487 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:55.499 INFO:teuthology.orchestra.run.smithi032.stdout:5255 write 0x43b68b9 thru 0x43c1840 (0xaf88 bytes) 2024-09-13T09:18:55.507 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:55.509 INFO:teuthology.orchestra.run.smithi032.stdout:5256 write 0x92f0714 thru 0x92f7005 (0x68f2 bytes) 2024-09-13T09:18:55.520 INFO:teuthology.orchestra.run.smithi032.stdout:5257 trunc from 0xccdc093 to 0x9867c25 2024-09-13T09:18:55.524 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:55.524 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:55.531 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:55.526+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:55.534 INFO:teuthology.orchestra.run.smithi032.stdout:5258 punch from 0x6e4029f to 0x6e43fe5, (0x3d46 bytes) 2024-09-13T09:18:55.534 INFO:teuthology.orchestra.run.smithi032.stdout:5259 read 0x1bd8b44 thru 0x1bddea0 (0x535d bytes) 2024-09-13T09:18:55.535 INFO:teuthology.orchestra.run.smithi032.stdout:5260 read 0x6b76d0c thru 0x6b81ade (0xadd3 bytes) 2024-09-13T09:18:55.542 INFO:teuthology.orchestra.run.smithi032.stdout:5262 trunc from 0x9867c25 to 0xd35a2a3 2024-09-13T09:18:55.542 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:55.542 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:55.551 INFO:teuthology.orchestra.run.smithi032.stdout:5263 write 0x819d64f thru 0x81abea0 (0xe852 bytes) 2024-09-13T09:18:55.558 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:55.559 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:55.559 INFO:teuthology.orchestra.run.smithi032.stdout:5265 read 0x9f6b07a thru 0x9f6ca6f (0x19f6 bytes) 2024-09-13T09:18:55.559 INFO:teuthology.orchestra.run.smithi032.stdout:5267 punch from 0xa42bec5 to 0xa439532, (0xd66d bytes) 2024-09-13T09:18:55.572 INFO:teuthology.orchestra.run.smithi032.stdout:5269 trunc from 0xd35a2a3 to 0x57e8f63 2024-09-13T09:18:55.577 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:55.599 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:55.609 INFO:teuthology.orchestra.run.smithi032.stdout:5272 trunc from 0x57e8f63 to 0x1f50d 2024-09-13T09:18:55.623 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:55.623 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:55.631 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:55.629+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:55.645 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:55.645 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:55.646 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:55.666 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:18:55.669 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:55.669 INFO:teuthology.orchestra.run.smithi032.stdout:5273 write 0x69e4c21 thru 0x69e8ee5 (0x42c5 bytes) 2024-09-13T09:18:55.669 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:55.676 INFO:teuthology.orchestra.run.smithi032.stdout:5274 write 0x1a63a7d thru 0x1a6cf15 (0x9499 bytes) 2024-09-13T09:18:55.682 INFO:teuthology.orchestra.run.smithi032.stdout:5275 read 0x43e44 thru 0x4c055 (0x8212 bytes) 2024-09-13T09:18:55.683 INFO:teuthology.orchestra.run.smithi032.stdout:5276 write 0x59ecf0 thru 0x5a19fd (0x2d0e bytes) 2024-09-13T09:18:55.689 INFO:teuthology.orchestra.run.smithi032.stdout:5277 read 0x49508ad thru 0x495daad (0xd201 bytes) 2024-09-13T09:18:55.707 INFO:teuthology.orchestra.run.smithi032.stdout:5278 write 0x9a8e081 thru 0x9a98196 (0xa116 bytes) 2024-09-13T09:18:55.728 INFO:teuthology.orchestra.run.smithi032.stdout:5279 write 0xb6a6fca thru 0xb6aa4b1 (0x34e8 bytes) 2024-09-13T09:18:55.732 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:55.726+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:55.735 INFO:teuthology.orchestra.run.smithi032.stdout:5280 trunc from 0xb6aa4b2 to 0xc575a1f 2024-09-13T09:18:55.746 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:55.746 INFO:teuthology.orchestra.run.smithi032.stdout:5283 read 0x1e75ce4 thru 0x1e84c53 (0xef70 bytes) 2024-09-13T09:18:55.750 INFO:teuthology.orchestra.run.smithi032.stdout:5286 trunc from 0xc575a1f to 0xa87b887 2024-09-13T09:18:55.766 INFO:teuthology.orchestra.run.smithi032.stdout:5289 read 0x6dc124b thru 0x6dcb389 (0xa13f bytes) 2024-09-13T09:18:55.766 INFO:teuthology.orchestra.run.smithi032.stdout:5290 write 0x3865205 thru 0x386c147 (0x6f43 bytes) 2024-09-13T09:18:55.774 INFO:teuthology.orchestra.run.smithi032.stdout:5292 punch from 0x92cea61 to 0x92ddfe9, (0xf588 bytes) 2024-09-13T09:18:55.774 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:55.774 INFO:teuthology.orchestra.run.smithi032.stdout:5295 read 0x44c6ee5 thru 0x44d1e3e (0xaf5a bytes) 2024-09-13T09:18:55.774 INFO:teuthology.orchestra.run.smithi032.stdout:5296 read 0x67db393 thru 0x67e056c (0x51da bytes) 2024-09-13T09:18:55.774 INFO:teuthology.orchestra.run.smithi032.stdout:5297 write 0x5145e63 thru 0x5150e9a (0xb038 bytes) 2024-09-13T09:18:55.794 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:55.795 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:55.797 INFO:teuthology.orchestra.run.smithi032.stdout:5299 writesame 0xb9aaefe thru 0xb9b90a5 data_size 0x74(0xe1a8 bytes) 2024-09-13T09:18:55.801 INFO:teuthology.orchestra.run.smithi032.stdout:5300 trunc from 0xb9b90a6 to 0xd34a540 2024-09-13T09:18:55.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:18:55.816 INFO:teuthology.orchestra.run.smithi032.stdout:5301 read 0xac6e88e thru 0xac73228 (0x499b bytes) 2024-09-13T09:18:55.830 INFO:teuthology.orchestra.run.smithi032.stdout:5302 write 0xde0c8f4 thru 0xde1b83f (0xef4c bytes) 2024-09-13T09:18:55.833 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:55.829+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:55.859 INFO:teuthology.orchestra.run.smithi032.stdout:5304 trunc from 0xde1b840 to 0x64e754 2024-09-13T09:18:55.924 INFO:teuthology.orchestra.run.smithi032.stdout:5305 writesame 0xbd8c08e thru 0xbd94432 data_size 0x1(0x83a5 bytes) 2024-09-13T09:18:55.924 INFO:teuthology.orchestra.run.smithi032.stdout:5308 write 0x34d9fa0 thru 0x34e3d4a (0x9dab bytes) 2024-09-13T09:18:55.931 INFO:teuthology.orchestra.run.smithi032.stdout:5309 read 0x2814e44 thru 0x281eea1 (0xa05e bytes) 2024-09-13T09:18:55.932 INFO:teuthology.orchestra.run.smithi032.stdout:5312 write 0x1286629 thru 0x1290667 (0xa03f bytes) 2024-09-13T09:18:55.933 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:55.929+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:55.936 INFO:teuthology.orchestra.run.smithi032.stdout:5313 read 0x924e903 thru 0x9256de8 (0x84e6 bytes) 2024-09-13T09:18:55.937 INFO:teuthology.orchestra.run.smithi032.stdout:5314 write 0x80108b1 thru 0x801f529 (0xec79 bytes) 2024-09-13T09:18:55.945 INFO:teuthology.orchestra.run.smithi032.stdout:5316 writesame 0xb481550 thru 0xb48bf22 data_size 0x2f(0xa9d3 bytes) 2024-09-13T09:18:55.952 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:55.953 INFO:teuthology.orchestra.run.smithi032.stdout:5318 writesame 0x2634b80 thru 0x26350ad data_size 0x6(0x52e bytes) 2024-09-13T09:18:55.956 INFO:teuthology.orchestra.run.smithi032.stdout:5319 write 0x871bc07 thru 0x87291ae (0xd5a8 bytes) 2024-09-13T09:18:55.959 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:55.960 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:55.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:18:55.979 INFO:teuthology.orchestra.run.smithi032.stdout:5320 write 0xd51251a thru 0xd51d682 (0xb169 bytes) 2024-09-13T09:18:55.985 INFO:teuthology.orchestra.run.smithi032.stdout:5321 punch from 0x7e31b5b to 0x7e3c8bf, (0xad64 bytes) 2024-09-13T09:18:55.998 INFO:teuthology.orchestra.run.smithi032.stdout:5323 write 0xe97ec2d thru 0xe985047 (0x641b bytes) 2024-09-13T09:18:56.004 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.004 INFO:teuthology.orchestra.run.smithi032.stdout:5326 read 0x157cc1c thru 0x15858e0 (0x8cc5 bytes) 2024-09-13T09:18:56.004 INFO:teuthology.orchestra.run.smithi032.stdout:5327 read 0x52ece83 thru 0x52f0d83 (0x3f01 bytes) 2024-09-13T09:18:56.004 INFO:teuthology.orchestra.run.smithi032.stdout:5328 read 0x64f569f thru 0x64f74c9 (0x1e2b bytes) 2024-09-13T09:18:56.004 INFO:teuthology.orchestra.run.smithi032.stdout:5329 write 0xbf5c82b thru 0xbf5da69 (0x123f bytes) 2024-09-13T09:18:56.010 INFO:teuthology.orchestra.run.smithi032.stdout:5330 punch from 0x2824a29 to 0x28347af, (0xfd86 bytes) 2024-09-13T09:18:56.011 INFO:teuthology.orchestra.run.smithi032.stdout:5332 writesame 0xabb1652 thru 0xabc07a7 data_size 0x15(0xf156 bytes) 2024-09-13T09:18:56.017 INFO:teuthology.orchestra.run.smithi032.stdout:5333 write 0x54a064a thru 0x54a2345 (0x1cfc bytes) 2024-09-13T09:18:56.034 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:56.030+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:56.039 INFO:teuthology.orchestra.run.smithi032.stdout:5336 trunc from 0xe985048 to 0x56cc441 2024-09-13T09:18:56.072 INFO:teuthology.orchestra.run.smithi032.stdout:5337 punch from 0xfd307b to 0xfe224e, (0xf1d3 bytes) 2024-09-13T09:18:56.073 INFO:teuthology.orchestra.run.smithi032.stdout:5339 read 0x38d60f5 thru 0x38e4dfe (0xed0a bytes) 2024-09-13T09:18:56.104 INFO:teuthology.orchestra.run.smithi032.stdout:5341 writesame 0xdbffa22 thru 0xdc0f076 data_size 0x1(0xf655 bytes) 2024-09-13T09:18:56.104 INFO:teuthology.orchestra.run.smithi032.stdout:5343 punch from 0x6a34786 to 0x6a3b2df, (0x6b59 bytes) 2024-09-13T09:18:56.105 INFO:teuthology.orchestra.run.smithi032.stdout:5344 read 0x2afab28 thru 0x2b0181e (0x6cf7 bytes) 2024-09-13T09:18:56.105 INFO:teuthology.orchestra.run.smithi032.stdout:5345 punch from 0x5fb503a to 0x5fbd22c, (0x81f2 bytes) 2024-09-13T09:18:56.105 INFO:teuthology.orchestra.run.smithi032.stdout:5346 writesame 0xcdf320e thru 0xcdfbcaf data_size 0x3(0x8aa2 bytes) 2024-09-13T09:18:56.124 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:56.124 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:56.135 INFO:teuthology.orchestra.run.smithi032.stdout:5347 trunc from 0xdc0f077 to 0x6035d8e 2024-09-13T09:18:56.136 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:56.133+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:56.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:18:56.163 INFO:teuthology.orchestra.run.smithi032.stdout:5348 read 0x151769a thru 0x151dce8 (0x664f bytes) 2024-09-13T09:18:56.183 INFO:teuthology.orchestra.run.smithi032.stdout:5349 write 0x8ae6061 thru 0x8ae8f0f (0x2eaf bytes) 2024-09-13T09:18:56.189 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.189 INFO:teuthology.orchestra.run.smithi032.stdout:5352 read 0x76f5213 thru 0x7702398 (0xd186 bytes) 2024-09-13T09:18:56.189 INFO:teuthology.orchestra.run.smithi032.stdout:5353 writesame 0x8449ff9 thru 0x844fe81 data_size 0x3(0x5e89 bytes) 2024-09-13T09:18:56.207 INFO:teuthology.orchestra.run.smithi032.stdout:5354 trunc from 0x8ae8f10 to 0x6cbd72f 2024-09-13T09:18:56.224 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.224 INFO:teuthology.orchestra.run.smithi032.stdout:5356 read 0x1aa6afd thru 0x1ab0d08 (0xa20c bytes) 2024-09-13T09:18:56.224 INFO:teuthology.orchestra.run.smithi032.stdout:5357 read 0xfc5cd6 thru 0xfcb759 (0x5a84 bytes) 2024-09-13T09:18:56.224 INFO:teuthology.orchestra.run.smithi032.stdout:5358 read 0xf0f4bd thru 0xf17345 (0x7e89 bytes) 2024-09-13T09:18:56.236 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:56.233+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:56.251 INFO:teuthology.orchestra.run.smithi032.stdout:5359 write 0xe67e047 thru 0xe67e610 (0x5ca bytes) 2024-09-13T09:18:56.257 INFO:teuthology.orchestra.run.smithi032.stdout:5360 read 0x1356856 thru 0x1360b2f (0xa2da bytes) 2024-09-13T09:18:56.258 INFO:teuthology.orchestra.run.smithi032.stdout:5361 writesame 0x3a94d26 thru 0x3aa1709 data_size 0x49(0xc9e4 bytes) 2024-09-13T09:18:56.265 INFO:teuthology.orchestra.run.smithi032.stdout:5362 write 0x9356aae thru 0x9360c77 (0xa1ca bytes) 2024-09-13T09:18:56.272 INFO:teuthology.orchestra.run.smithi032.stdout:5363 punch from 0x3776f39 to 0x377a8d6, (0x399d bytes) 2024-09-13T09:18:56.280 INFO:teuthology.orchestra.run.smithi032.stdout:5364 trunc from 0xe67e611 to 0x9ff507b 2024-09-13T09:18:56.315 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:56.315 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:56.321 INFO:teuthology.orchestra.run.smithi032.stdout:5365 write 0xb8ab515 thru 0xb8b51cb (0x9cb7 bytes) 2024-09-13T09:18:56.328 INFO:teuthology.orchestra.run.smithi032.stdout:5367 trunc from 0xb8b51cc to 0xb392062 2024-09-13T09:18:56.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:18:56.342 INFO:teuthology.orchestra.run.smithi032.stdout:5368 read 0x834f5da thru 0x835a06a (0xaa91 bytes) 2024-09-13T09:18:56.343 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:56.337+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:56.345 INFO:teuthology.orchestra.run.smithi032.stdout:5370 trunc from 0xb392062 to 0xccd2bbc 2024-09-13T09:18:56.357 INFO:teuthology.orchestra.run.smithi032.stdout:5371 read 0x7e9c6ca thru 0x7ea1ed3 (0x580a bytes) 2024-09-13T09:18:56.357 INFO:teuthology.orchestra.run.smithi032.stdout:5372 write 0x8d3acbd thru 0x8d4a655 (0xf999 bytes) 2024-09-13T09:18:56.364 INFO:teuthology.orchestra.run.smithi032.stdout:5374 write 0x2b9b116 thru 0x2b9b86c (0x757 bytes) 2024-09-13T09:18:56.366 INFO:teuthology.orchestra.run.smithi032.stdout:5375 write 0xcff767 thru 0xd0c142 (0xc9dc bytes) 2024-09-13T09:18:56.387 INFO:teuthology.orchestra.run.smithi032.stdout:5376 trunc from 0xccd2bbc to 0x41ca29f 2024-09-13T09:18:56.411 INFO:teuthology.orchestra.run.smithi032.stdout:5379 read 0x28734a0 thru 0x287f5f8 (0xc159 bytes) 2024-09-13T09:18:56.412 INFO:teuthology.orchestra.run.smithi032.stdout:5380 punch from 0x2bdc535 to 0x2be74a2, (0xaf6d bytes) 2024-09-13T09:18:56.412 INFO:teuthology.orchestra.run.smithi032.stdout:5381 read 0x6cfe90 thru 0x6d7c61 (0x7dd2 bytes) 2024-09-13T09:18:56.412 INFO:teuthology.orchestra.run.smithi032.stdout:5382 read 0xe9e70d thru 0xeacb91 (0xe485 bytes) 2024-09-13T09:18:56.413 INFO:teuthology.orchestra.run.smithi032.stdout:5384 writesame 0x19b985f thru 0x19c7ba2 data_size 0xa(0xe344 bytes) 2024-09-13T09:18:56.439 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:56.433+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:56.446 INFO:teuthology.orchestra.run.smithi032.stdout:5385 write 0x740ea09 thru 0x7419777 (0xad6f bytes) 2024-09-13T09:18:56.453 INFO:teuthology.orchestra.run.smithi032.stdout:5386 writesame 0x704b0c thru 0x70a383 data_size 0x26(0x5878 bytes) 2024-09-13T09:18:56.457 INFO:teuthology.orchestra.run.smithi032.stdout:5388 read 0x2a4eb69 thru 0x2a54322 (0x57ba bytes) 2024-09-13T09:18:56.457 INFO:teuthology.orchestra.run.smithi032.stdout:5389 punch from 0x43ffe16 to 0x4401c6e, (0x1e58 bytes) 2024-09-13T09:18:56.473 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:56.474 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:56.488 INFO:teuthology.orchestra.run.smithi032.stdout:5390 write 0xd4a3505 thru 0xd4b1da1 (0xe89d bytes) 2024-09-13T09:18:56.489 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:18:56.494 INFO:teuthology.orchestra.run.smithi032.stdout:5391 write 0x1013694 thru 0x1018b66 (0x54d3 bytes) 2024-09-13T09:18:56.499 INFO:teuthology.orchestra.run.smithi032.stdout:5392 trunc from 0xd4b1da2 to 0xd0eb61c 2024-09-13T09:18:56.530 INFO:teuthology.orchestra.run.smithi032.stdout:5393 writesame 0xda777e3 thru 0xda7b77b data_size 0x1b(0x3f99 bytes) 2024-09-13T09:18:56.530 INFO:teuthology.orchestra.run.smithi032.stdout:5394 punch from 0xfad223 to 0xfbaeec, (0xdcc9 bytes) 2024-09-13T09:18:56.530 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.530 INFO:teuthology.orchestra.run.smithi032.stdout:5399 read 0x8cf7390 thru 0x8cfb66a (0x42db bytes) 2024-09-13T09:18:56.530 INFO:teuthology.orchestra.run.smithi032.stdout:5400 punch from 0xad46807 to 0xad55cc2, (0xf4bb bytes) 2024-09-13T09:18:56.530 INFO:teuthology.orchestra.run.smithi032.stdout:5401 read 0x68f5d76 thru 0x68f910f (0x339a bytes) 2024-09-13T09:18:56.530 INFO:teuthology.orchestra.run.smithi032.stdout:5402 write 0xa9f1d00 thru 0xa9fe205 (0xc506 bytes) 2024-09-13T09:18:56.535 INFO:teuthology.orchestra.run.smithi032.stdout:5403 punch from 0xcc82048 to 0xcc82b9a, (0xb52 bytes) 2024-09-13T09:18:56.535 INFO:teuthology.orchestra.run.smithi032.stdout:5405 read 0xcbd1046 thru 0xcbd8368 (0x7323 bytes) 2024-09-13T09:18:56.535 INFO:teuthology.orchestra.run.smithi032.stdout:5406 writesame 0x62e0aac thru 0x62ee7fd data_size 0x47(0xdd52 bytes) 2024-09-13T09:18:56.540 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:56.534+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:56.543 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.543 INFO:teuthology.orchestra.run.smithi032.stdout:5411 write 0x7d06a0e thru 0x7d16357 (0xf94a bytes) 2024-09-13T09:18:56.549 INFO:teuthology.orchestra.run.smithi032.stdout:5412 read 0xcf712a2 thru 0xcf7bea0 (0xabff bytes) 2024-09-13T09:18:56.550 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.550 INFO:teuthology.orchestra.run.smithi032.stdout:5414 write 0x236259a thru 0x236839d (0x5e04 bytes) 2024-09-13T09:18:56.555 INFO:teuthology.orchestra.run.smithi032.stdout:5416 read 0x2b7d387 thru 0x2b8b297 (0xdf11 bytes) 2024-09-13T09:18:56.556 INFO:teuthology.orchestra.run.smithi032.stdout:5417 read 0xacf544d thru 0xad01528 (0xc0dc bytes) 2024-09-13T09:18:56.556 INFO:teuthology.orchestra.run.smithi032.stdout:5418 writesame 0xd66cdb8 thru 0xd67c1d7 data_size 0x7e(0xf420 bytes) 2024-09-13T09:18:56.558 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.558 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.558 INFO:teuthology.orchestra.run.smithi032.stdout:5422 punch from 0x862286b to 0x8626a63, (0x41f8 bytes) 2024-09-13T09:18:56.558 INFO:teuthology.orchestra.run.smithi032.stdout:5423 writesame 0x7f64dee thru 0x7f6c2ed data_size 0x1a(0x7500 bytes) 2024-09-13T09:18:56.562 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.575 INFO:teuthology.orchestra.run.smithi032.stdout:5426 write 0xde50459 thru 0xde54990 (0x4538 bytes) 2024-09-13T09:18:56.583 INFO:teuthology.orchestra.run.smithi032.stdout:5427 read 0xb8ba557 thru 0xb8bc5cf (0x2079 bytes) 2024-09-13T09:18:56.583 INFO:teuthology.orchestra.run.smithi032.stdout:5430 read 0x5f4190d thru 0x5f49a5a (0x814e bytes) 2024-09-13T09:18:56.583 INFO:teuthology.orchestra.run.smithi032.stdout:5431 read 0xc45b3fb thru 0xc46a07e (0xec84 bytes) 2024-09-13T09:18:56.583 INFO:teuthology.orchestra.run.smithi032.stdout:5432 punch from 0xab5c2d3 to 0xab673a3, (0xb0d0 bytes) 2024-09-13T09:18:56.584 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.584 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xed666bd 2024-09-13T09:18:56.584 INFO:teuthology.orchestra.run.smithi032.stdout:5434 trunc from 0xde54991 to 0xed666bd 2024-09-13T09:18:56.596 INFO:teuthology.orchestra.run.smithi032.stdout:5435 read 0x6431890 thru 0x6434ff2 (0x3763 bytes) 2024-09-13T09:18:56.597 INFO:teuthology.orchestra.run.smithi032.stdout:5436 read 0x8cb2178 thru 0x8cb4749 (0x25d2 bytes) 2024-09-13T09:18:56.597 INFO:teuthology.orchestra.run.smithi032.stdout:5437 read 0x6f2f301 thru 0x6f3287c (0x357c bytes) 2024-09-13T09:18:56.597 INFO:teuthology.orchestra.run.smithi032.stdout:5439 read 0x64debb8 thru 0x64ec377 (0xd7c0 bytes) 2024-09-13T09:18:56.597 INFO:teuthology.orchestra.run.smithi032.stdout:5440 punch from 0x603fadb to 0x604afbc, (0xb4e1 bytes) 2024-09-13T09:18:56.597 INFO:teuthology.orchestra.run.smithi032.stdout:5441 read 0x4e1455f thru 0x4e1ad74 (0x6816 bytes) 2024-09-13T09:18:56.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:18:56.616 INFO:teuthology.orchestra.run.smithi032.stdout:5442 trunc from 0xed666bd to 0x26cc719 2024-09-13T09:18:56.634 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:56.634 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:56.641 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:56.634+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:56.643 INFO:teuthology.orchestra.run.smithi032.stdout:5444 read 0x208a333 thru 0x209a048 (0xfd16 bytes) 2024-09-13T09:18:56.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:18:56.664 INFO:teuthology.orchestra.run.smithi032.stdout:5445 writesame 0x754c3ec thru 0x755b1f8 data_size 0x1(0xee0d bytes) 2024-09-13T09:18:56.664 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.664 INFO:teuthology.orchestra.run.smithi032.stdout:5449 read 0x3d9d510 thru 0x3daae89 (0xd97a bytes) 2024-09-13T09:18:56.665 INFO:teuthology.orchestra.run.smithi032.stdout:5451 write 0x23a27d7 thru 0x23af8f5 (0xd11f bytes) 2024-09-13T09:18:56.667 INFO:teuthology.orchestra.run.smithi032.stdout:5452 read 0x68dfe6a thru 0x68e09ca (0xb61 bytes) 2024-09-13T09:18:56.667 INFO:teuthology.orchestra.run.smithi032.stdout:5453 write 0x3ad2361 thru 0x3ad26dd (0x37d bytes) 2024-09-13T09:18:56.678 INFO:teuthology.orchestra.run.smithi032.stdout:5455 trunc from 0x755b1f9 to 0x135e621 2024-09-13T09:18:56.720 INFO:teuthology.orchestra.run.smithi032.stdout:5456 trunc from 0x135e621 to 0xd242e88 2024-09-13T09:18:56.731 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.731 INFO:teuthology.orchestra.run.smithi032.stdout:5461 read 0xab6cc07 thru 0xab6effa (0x23f4 bytes) 2024-09-13T09:18:56.732 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.733 INFO:teuthology.orchestra.run.smithi032.stdout:5464 trunc from 0xd242e88 to 0xc2cb889 2024-09-13T09:18:56.741 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:56.737+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:56.744 INFO:teuthology.orchestra.run.smithi032.stdout:5466 read 0x271e78c thru 0x2728b98 (0xa40d bytes) 2024-09-13T09:18:56.745 INFO:teuthology.orchestra.run.smithi032.stdout:5467 write 0x1c94b46 thru 0x1c9ef72 (0xa42d bytes) 2024-09-13T09:18:56.751 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.752 INFO:teuthology.orchestra.run.smithi032.stdout:5469 read 0x2c92c57 thru 0x2ca198e (0xed38 bytes) 2024-09-13T09:18:56.762 INFO:teuthology.orchestra.run.smithi032.stdout:5470 write 0xc4a2cad thru 0xc4a939b (0x66ef bytes) 2024-09-13T09:18:56.768 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.768 INFO:teuthology.orchestra.run.smithi032.stdout:5475 write 0x987db42 thru 0x9880598 (0x2a57 bytes) 2024-09-13T09:18:56.775 INFO:teuthology.orchestra.run.smithi032.stdout:5476 punch from 0x90e35cd to 0x90ea67d, (0x70b0 bytes) 2024-09-13T09:18:56.775 INFO:teuthology.orchestra.run.smithi032.stdout:5477 punch from 0x161ebc to 0x1633f5, (0x1539 bytes) 2024-09-13T09:18:56.775 INFO:teuthology.orchestra.run.smithi032.stdout:5479 read 0x30741cf thru 0x3079089 (0x4ebb bytes) 2024-09-13T09:18:56.795 INFO:teuthology.orchestra.run.smithi032.stdout:5480 trunc from 0xc4a939c to 0xd86ded 2024-09-13T09:18:56.808 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:56.808 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:56.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:18:56.829 INFO:teuthology.orchestra.run.smithi032.stdout:5481 read 0x3b503 thru 0x41c94 (0x6792 bytes) 2024-09-13T09:18:56.830 INFO:teuthology.orchestra.run.smithi032.stdout:5482 punch from 0x95a49 to 0x9625f, (0x816 bytes) 2024-09-13T09:18:56.830 INFO:teuthology.orchestra.run.smithi032.stdout:5484 read 0x1f7368 thru 0x1fd0e2 (0x5d7b bytes) 2024-09-13T09:18:56.831 INFO:teuthology.orchestra.run.smithi032.stdout:5486 punch from 0x724a20 to 0x72bfc2, (0x75a2 bytes) 2024-09-13T09:18:56.842 INFO:teuthology.orchestra.run.smithi032.stdout:5487 trunc from 0xd86ded to 0x837ee3f 2024-09-13T09:18:56.842 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:56.838+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:56.853 INFO:teuthology.orchestra.run.smithi032.stdout:5488 write 0x1c4c846 thru 0x1c54029 (0x77e4 bytes) 2024-09-13T09:18:56.860 INFO:teuthology.orchestra.run.smithi032.stdout:5489 read 0x17fa621 thru 0x17fc20f (0x1bef bytes) 2024-09-13T09:18:56.860 INFO:teuthology.orchestra.run.smithi032.stdout:5490 punch from 0x1faee83 to 0x1fb0eaa, (0x2027 bytes) 2024-09-13T09:18:56.860 INFO:teuthology.orchestra.run.smithi032.stdout:5491 read 0x3b753d3 thru 0x3b765ff (0x122d bytes) 2024-09-13T09:18:56.860 INFO:teuthology.orchestra.run.smithi032.stdout:5492 write 0x38443b4 thru 0x384ee91 (0xaade bytes) 2024-09-13T09:18:56.884 INFO:teuthology.orchestra.run.smithi032.stdout:5493 writesame 0xc16a120 thru 0xc179bb7 data_size 0x84(0xfa98 bytes) 2024-09-13T09:18:56.884 INFO:teuthology.orchestra.run.smithi032.stdout:5494 read 0x7545834 thru 0x75533d5 (0xdba2 bytes) 2024-09-13T09:18:56.884 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:56.884 INFO:teuthology.orchestra.run.smithi032.stdout:5497 writesame 0x4a82ccd thru 0x4a8418e data_size 0x1(0x14c2 bytes) 2024-09-13T09:18:56.915 INFO:teuthology.orchestra.run.smithi032.stdout:5498 writesame 0xec0d002 thru 0xec11b77 data_size 0xd(0x4b76 bytes) 2024-09-13T09:18:56.926 INFO:teuthology.orchestra.run.smithi032.stdout:5499 trunc from 0xec11b78 to 0x977a22a 2024-09-13T09:18:56.944 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:56.938+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:56.951 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:56.951 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:56.953 INFO:teuthology.orchestra.run.smithi032.stdout:5500 trunc from 0x977a22a to 0x2fddd8c 2024-09-13T09:18:56.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:18:56.980 INFO:teuthology.orchestra.run.smithi032.stdout:5502 read 0x24ff616 thru 0x2502165 (0x2b50 bytes) 2024-09-13T09:18:56.984 INFO:teuthology.orchestra.run.smithi032.stdout:5503 read 0x262e93c thru 0x262ea79 (0x13e bytes) 2024-09-13T09:18:56.985 INFO:teuthology.orchestra.run.smithi032.stdout:5505 punch from 0x1c17ea6 to 0x1c1886a, (0x9c4 bytes) 2024-09-13T09:18:56.992 INFO:teuthology.orchestra.run.smithi032.stdout:5506 trunc from 0x2fddd8c to 0x9f6fb07 2024-09-13T09:18:57.004 INFO:teuthology.orchestra.run.smithi032.stdout:5507 write 0x38b8f05 thru 0x38c3b88 (0xac84 bytes) 2024-09-13T09:18:57.007 INFO:teuthology.orchestra.run.smithi032.stdout:5508 write 0x848642f thru 0x848d7d0 (0x73a2 bytes) 2024-09-13T09:18:57.014 INFO:teuthology.orchestra.run.smithi032.stdout:5509 write 0x5bf4ac6 thru 0x5bf6f31 (0x246c bytes) 2024-09-13T09:18:57.032 INFO:teuthology.orchestra.run.smithi032.stdout:5510 trunc from 0x9f6fb07 to 0x25d749d 2024-09-13T09:18:57.045 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:57.038+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:57.072 INFO:teuthology.orchestra.run.smithi032.stdout:5511 trunc from 0x25d749d to 0xb15bd14 2024-09-13T09:18:57.085 INFO:teuthology.orchestra.run.smithi032.stdout:5512 punch from 0x55040cc to 0x5507a68, (0x399c bytes) 2024-09-13T09:18:57.085 INFO:teuthology.orchestra.run.smithi032.stdout:5513 write 0xab8b88b thru 0xab91ff6 (0x676c bytes) 2024-09-13T09:18:57.098 INFO:teuthology.orchestra.run.smithi032.stdout:5514 trunc from 0xb15bd14 to 0xe37c8aa 2024-09-13T09:18:57.110 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:57.110 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:57.112 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.112 INFO:teuthology.orchestra.run.smithi032.stdout:5516 punch from 0xccebc65 to 0xccfa33d, (0xe6d8 bytes) 2024-09-13T09:18:57.112 INFO:teuthology.orchestra.run.smithi032.stdout:5517 write 0x7569e72 thru 0x75752a0 (0xb42f bytes) 2024-09-13T09:18:57.120 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.120 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.120 INFO:teuthology.orchestra.run.smithi032.stdout:5524 writesame 0x3552741 thru 0x355c5d3 data_size 0x17(0x9e93 bytes) 2024-09-13T09:18:57.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:18:57.129 INFO:teuthology.orchestra.run.smithi032.stdout:5525 writesame 0xbf5fd67 thru 0xbf6c781 data_size 0x1(0xca1b bytes) 2024-09-13T09:18:57.175 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:57.173+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:57.197 INFO:teuthology.orchestra.run.smithi032.stdout:5527 read 0x7d05a7e thru 0x7d13ea0 (0xe423 bytes) 2024-09-13T09:18:57.197 INFO:teuthology.orchestra.run.smithi032.stdout:5528 read 0xe93980 thru 0xe98804 (0x4e85 bytes) 2024-09-13T09:18:57.209 INFO:teuthology.orchestra.run.smithi032.stdout:5529 trunc from 0xe37c8aa to 0x92e977e 2024-09-13T09:18:57.238 INFO:teuthology.orchestra.run.smithi032.stdout:5530 writesame 0x3ea9cb8 thru 0x3eb5f39 data_size 0x56(0xc282 bytes) 2024-09-13T09:18:57.246 INFO:teuthology.orchestra.run.smithi032.stdout:5532 trunc from 0x92e977e to 0x9781874 2024-09-13T09:18:57.246 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:57.242+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:57.259 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.262 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:57.262 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:57.280 INFO:teuthology.orchestra.run.smithi032.stdout:5534 trunc from 0x9781874 to 0x35e0d7 2024-09-13T09:18:57.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:18:57.348 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:57.342+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:57.348 INFO:teuthology.orchestra.run.smithi032.stdout:5535 trunc from 0x35e0d7 to 0xc4c18f0 2024-09-13T09:18:57.362 INFO:teuthology.orchestra.run.smithi032.stdout:5538 punch from 0x4525e87 to 0x452dc5f, (0x7dd8 bytes) 2024-09-13T09:18:57.363 INFO:teuthology.orchestra.run.smithi032.stdout:5539 write 0x9e77959 thru 0x9e7d390 (0x5a38 bytes) 2024-09-13T09:18:57.370 INFO:teuthology.orchestra.run.smithi032.stdout:5540 writesame 0x6e35fd3 thru 0x6e39d75 data_size 0x1(0x3da3 bytes) 2024-09-13T09:18:57.396 INFO:teuthology.orchestra.run.smithi032.stdout:5542 read 0xa1036af thru 0xa10a515 (0x6e67 bytes) 2024-09-13T09:18:57.396 INFO:teuthology.orchestra.run.smithi032.stdout:5543 read 0x526ac95 thru 0x5279446 (0xe7b2 bytes) 2024-09-13T09:18:57.396 INFO:teuthology.orchestra.run.smithi032.stdout:5544 read 0x22ad019 thru 0x22b6128 (0x9110 bytes) 2024-09-13T09:18:57.396 INFO:teuthology.orchestra.run.smithi032.stdout:5545 read 0x9bf6a78 thru 0x9c0195a (0xaee3 bytes) 2024-09-13T09:18:57.398 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.398 INFO:teuthology.orchestra.run.smithi032.stdout:5550 read 0x8a26150 thru 0x8a3256b (0xc41c bytes) 2024-09-13T09:18:57.398 INFO:teuthology.orchestra.run.smithi032.stdout:5551 punch from 0x85df5cf to 0x85e95d9, (0xa00a bytes) 2024-09-13T09:18:57.398 INFO:teuthology.orchestra.run.smithi032.stdout:5553 read 0x55bad2a thru 0x55bd94f (0x2c26 bytes) 2024-09-13T09:18:57.398 INFO:teuthology.orchestra.run.smithi032.stdout:5554 punch from 0xc1a1489 to 0xc1a1841, (0x3b8 bytes) 2024-09-13T09:18:57.398 INFO:teuthology.orchestra.run.smithi032.stdout:5557 read 0x57f220d thru 0x57f822f (0x6023 bytes) 2024-09-13T09:18:57.399 INFO:teuthology.orchestra.run.smithi032.stdout:5558 punch from 0x1346d95 to 0x13471e5, (0x450 bytes) 2024-09-13T09:18:57.399 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.399 INFO:teuthology.orchestra.run.smithi032.stdout:5560 punch from 0x55c815c to 0x55ccb3c, (0x49e0 bytes) 2024-09-13T09:18:57.399 INFO:teuthology.orchestra.run.smithi032.stdout:5561 writesame 0x5509edf thru 0x550f6e1 data_size 0x1(0x5803 bytes) 2024-09-13T09:18:57.417 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:57.417 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:57.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:18:57.441 INFO:teuthology.orchestra.run.smithi032.stdout:5563 write 0x11be3df thru 0x11c963d (0xb25f bytes) 2024-09-13T09:18:57.449 INFO:teuthology.orchestra.run.smithi032.stdout:5566 write 0x496a05d thru 0x49795a0 (0xf544 bytes) 2024-09-13T09:18:57.454 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.454 INFO:teuthology.orchestra.run.smithi032.stdout:5568 punch from 0x28c4263 to 0x28c5737, (0x14d4 bytes) 2024-09-13T09:18:57.454 INFO:teuthology.orchestra.run.smithi032.stdout:5575 writesame 0xb716e2b thru 0xb717bd9 data_size 0x1(0xdaf bytes) 2024-09-13T09:18:57.462 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.462 INFO:teuthology.orchestra.run.smithi032.stdout:5579 writesame 0xb460be8 thru 0xb466a8d data_size 0x5(0x5ea6 bytes) 2024-09-13T09:18:57.468 INFO:teuthology.orchestra.run.smithi032.stdout:5580 punch from 0x18b2b21 to 0x18b37cf, (0xcae bytes) 2024-09-13T09:18:57.468 INFO:teuthology.orchestra.run.smithi032.stdout:5582 punch from 0x87501b5 to 0x8756f75, (0x6dc0 bytes) 2024-09-13T09:18:57.468 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.468 INFO:teuthology.orchestra.run.smithi032.stdout:5585 write 0x81d45f0 thru 0x81e4460 (0xfe71 bytes) 2024-09-13T09:18:57.473 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:57.470+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:57.474 INFO:teuthology.orchestra.run.smithi032.stdout:5586 read 0x9974fd9 thru 0x9976b95 (0x1bbd bytes) 2024-09-13T09:18:57.481 INFO:teuthology.orchestra.run.smithi032.stdout:5587 trunc from 0xc4c18f0 to 0x9cf0228 2024-09-13T09:18:57.496 INFO:teuthology.orchestra.run.smithi032.stdout:5589 write 0x4f9ec99 thru 0x4fa0f1f (0x2287 bytes) 2024-09-13T09:18:57.502 INFO:teuthology.orchestra.run.smithi032.stdout:5590 punch from 0x31bca1a to 0x31cb7be, (0xeda4 bytes) 2024-09-13T09:18:57.502 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.529 INFO:teuthology.orchestra.run.smithi032.stdout:5592 write 0xeb9cf6e thru 0xeba6892 (0x9925 bytes) 2024-09-13T09:18:57.535 INFO:teuthology.orchestra.run.smithi032.stdout:5594 read 0x5a4511a thru 0x5a474d1 (0x23b8 bytes) 2024-09-13T09:18:57.536 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.550 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:57.545+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:57.569 INFO:teuthology.orchestra.run.smithi032.stdout:5596 trunc from 0xeba6893 to 0x150c62e 2024-09-13T09:18:57.586 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:18:57.603 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.603 INFO:teuthology.orchestra.run.smithi032.stdout:5598 read 0xeb17fe thru 0xeb8d94 (0x7597 bytes) 2024-09-13T09:18:57.604 INFO:teuthology.orchestra.run.smithi032.stdout:5599 read 0xfa7e84 thru 0xfad764 (0x58e1 bytes) 2024-09-13T09:18:57.607 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:57.607 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:57.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:18:57.627 INFO:teuthology.orchestra.run.smithi032.stdout:5600 writesame 0x8a952bc thru 0x8a9ab91 data_size 0x53(0x58d6 bytes) 2024-09-13T09:18:57.627 INFO:teuthology.orchestra.run.smithi032.stdout:5602 read 0x665bcdf thru 0x6665b46 (0x9e68 bytes) 2024-09-13T09:18:57.627 INFO:teuthology.orchestra.run.smithi032.stdout:5605 punch from 0x219571f to 0x2195ad2, (0x3b3 bytes) 2024-09-13T09:18:57.627 INFO:teuthology.orchestra.run.smithi032.stdout:5606 punch from 0x853badc to 0x8540742, (0x4c66 bytes) 2024-09-13T09:18:57.628 INFO:teuthology.orchestra.run.smithi032.stdout:5607 write 0x670e06d thru 0x670fbf0 (0x1b84 bytes) 2024-09-13T09:18:57.631 INFO:teuthology.orchestra.run.smithi032.stdout:5608 write 0x94cb1d thru 0x94ef87 (0x246b bytes) 2024-09-13T09:18:57.636 INFO:teuthology.orchestra.run.smithi032.stdout:5611 punch from 0x3f71f65 to 0x3f73fb2, (0x204d bytes) 2024-09-13T09:18:57.643 INFO:teuthology.orchestra.run.smithi032.stdout:5612 trunc from 0x8a9ab92 to 0xc4a9d80 2024-09-13T09:18:57.661 INFO:teuthology.orchestra.run.smithi032.stdout:5613 write 0xd72269a thru 0xd72cd09 (0xa670 bytes) 2024-09-13T09:18:57.665 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:57.662+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:57.666 INFO:teuthology.orchestra.run.smithi032.stdout:5616 read 0x164d9ae thru 0x165d8b7 (0xff0a bytes) 2024-09-13T09:18:57.676 INFO:teuthology.orchestra.run.smithi032.stdout:5617 writesame 0xe6645c9 thru 0xe6739a0 data_size 0x33(0xf3d8 bytes) 2024-09-13T09:18:57.676 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:18:57.690 INFO:teuthology.orchestra.run.smithi032.stdout:5620 trunc from 0xe6739a1 to 0x628c345 2024-09-13T09:18:57.711 INFO:teuthology.orchestra.run.smithi032.stdout:5621 read 0x571ca56 thru 0x572248f (0x5a3a bytes) 2024-09-13T09:18:57.711 INFO:teuthology.orchestra.run.smithi032.stdout:5622 read 0x52ca152 thru 0x52d3e76 (0x9d25 bytes) 2024-09-13T09:18:57.717 INFO:teuthology.orchestra.run.smithi032.stdout:5623 trunc from 0x628c345 to 0x2a907ef 2024-09-13T09:18:57.725 INFO:teuthology.orchestra.run.smithi032.stdout:5624 punch from 0xc69753 to 0xc7202d, (0x88da bytes) 2024-09-13T09:18:57.726 INFO:teuthology.orchestra.run.smithi032.stdout:5627 punch from 0x251ac11 to 0x25292ba, (0xe6a9 bytes) 2024-09-13T09:18:57.744 INFO:teuthology.orchestra.run.smithi032.stdout:5628 write 0x79eb710 thru 0x79edb87 (0x2478 bytes) 2024-09-13T09:18:57.751 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:57.745+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:57.757 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:57.757 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:57.768 INFO:teuthology.orchestra.run.smithi032.stdout:5629 write 0xc5788a6 thru 0xc57c2b3 (0x3a0e bytes) 2024-09-13T09:18:57.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:18:57.774 INFO:teuthology.orchestra.run.smithi032.stdout:5630 write 0x920966f thru 0x92177f2 (0xe184 bytes) 2024-09-13T09:18:57.798 INFO:teuthology.orchestra.run.smithi032.stdout:5631 trunc from 0xc57c2b4 to 0xd1cfdc 2024-09-13T09:18:57.822 INFO:teuthology.orchestra.run.smithi032.stdout:5633 read 0x2e2948 thru 0x2e543c (0x2af5 bytes) 2024-09-13T09:18:57.845 INFO:teuthology.orchestra.run.smithi032.stdout:5634 writesame 0x5f2f326 thru 0x5f3ce5c data_size 0x1(0xdb37 bytes) 2024-09-13T09:18:57.845 INFO:teuthology.orchestra.run.smithi032.stdout:5635 read 0x16894d8 thru 0x1694aa5 (0xb5ce bytes) 2024-09-13T09:18:57.845 INFO:teuthology.orchestra.run.smithi032.stdout:5637 read 0x564d6bc thru 0x5652a2a (0x536f bytes) 2024-09-13T09:18:57.845 INFO:teuthology.orchestra.run.smithi032.stdout:5638 punch from 0x317ea7c to 0x3181265, (0x27e9 bytes) 2024-09-13T09:18:57.845 INFO:teuthology.orchestra.run.smithi032.stdout:5639 read 0x12723b7 thru 0x1275612 (0x325c bytes) 2024-09-13T09:18:57.845 INFO:teuthology.orchestra.run.smithi032.stdout:5641 punch from 0x4a41e0c to 0x4a4b04d, (0x9241 bytes) 2024-09-13T09:18:57.845 INFO:teuthology.orchestra.run.smithi032.stdout:5642 punch from 0x5686ad4 to 0x568bd01, (0x522d bytes) 2024-09-13T09:18:57.866 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:57.862+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:57.866 INFO:teuthology.orchestra.run.smithi032.stdout:5643 write 0xbd3fd0b thru 0xbd44d09 (0x4fff bytes) 2024-09-13T09:18:57.871 INFO:teuthology.orchestra.run.smithi032.stdout:5644 write 0x377db5e thru 0x378a6f1 (0xcb94 bytes) 2024-09-13T09:18:57.875 INFO:teuthology.orchestra.run.smithi032.stdout:5645 trunc from 0xbd44d0a to 0xbc65be9 2024-09-13T09:18:57.876 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:18:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:18:57.888 INFO:teuthology.orchestra.run.smithi032.stdout:5646 write 0x1c099bb thru 0x1c168ec (0xcf32 bytes) 2024-09-13T09:18:57.893 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.894 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:18:57.894 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:18:57.895 INFO:teuthology.orchestra.run.smithi032.stdout:5649 trunc from 0xbc65be9 to 0xcfc13e4 2024-09-13T09:18:57.905 INFO:teuthology.orchestra.run.smithi032.stdout:5650 punch from 0x355192d to 0x355b1a3, (0x9876 bytes) 2024-09-13T09:18:57.905 INFO:teuthology.orchestra.run.smithi032.stdout:5651 read 0x4ae39dd thru 0x4ae72fe (0x3922 bytes) 2024-09-13T09:18:57.914 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:18:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:18:57.920 INFO:teuthology.orchestra.run.smithi032.stdout:5652 trunc from 0xcfc13e4 to 0x4399d3a 2024-09-13T09:18:57.932 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:18:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:18:57.943 INFO:teuthology.orchestra.run.smithi032.stdout:5654 read 0x1e90e2f thru 0x1ea0ba6 (0xfd78 bytes) 2024-09-13T09:18:57.945 INFO:teuthology.orchestra.run.smithi032.stdout:5655 punch from 0x2d64a9d to 0x2d6b1c3, (0x6726 bytes) 2024-09-13T09:18:57.945 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:57.949 INFO:teuthology.orchestra.run.smithi032.stdout:5657 trunc from 0x4399d3a to 0x657a858 2024-09-13T09:18:57.952 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:57.949+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:57.953 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:18:57.954 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:18:57.959 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:18:57.959 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:18:57.960 INFO:teuthology.orchestra.run.smithi032.stdout:5658 read 0x356599b thru 0x356cadd (0x7143 bytes) 2024-09-13T09:18:57.969 INFO:teuthology.orchestra.run.smithi032.stdout:5659 trunc from 0x657a858 to 0xb64048 2024-09-13T09:18:57.976 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:18:57.976 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:18:57.977 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:18:57.997 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:18:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:18:58.002 INFO:teuthology.orchestra.run.smithi032.stdout:5661 write 0x8f570ec thru 0x8f5ee5e (0x7d73 bytes) 2024-09-13T09:18:58.006 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.006 INFO:teuthology.orchestra.run.smithi032.stdout:5663 punch from 0x393dd87 to 0x39433a7, (0x5620 bytes) 2024-09-13T09:18:58.006 INFO:teuthology.orchestra.run.smithi032.stdout:5664 punch from 0x8a64f4b to 0x8a6d90b, (0x89c0 bytes) 2024-09-13T09:18:58.006 INFO:teuthology.orchestra.run.smithi032.stdout:5665 trunc from 0x8f5ee5f to 0x93af6a6 2024-09-13T09:18:58.018 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:18:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:18:58.030 INFO:teuthology.orchestra.run.smithi032.stdout:5668 writesame 0xe961b16 thru 0xe963e2f data_size 0x1(0x231a bytes) 2024-09-13T09:18:58.030 INFO:teuthology.orchestra.run.smithi032.stdout:5670 write 0x78a93f8 thru 0x78afb81 (0x678a bytes) 2024-09-13T09:18:58.035 INFO:teuthology.orchestra.run.smithi032.stdout:5671 punch from 0x6fd1e44 to 0x6fda610, (0x87cc bytes) 2024-09-13T09:18:58.036 INFO:teuthology.orchestra.run.smithi032.stdout:5672 read 0x1cb4e90 thru 0x1cb897e (0x3aef bytes) 2024-09-13T09:18:58.036 INFO:teuthology.orchestra.run.smithi032.stdout:5675 write 0xb3e28a6 thru 0xb3ed7a4 (0xaeff bytes) 2024-09-13T09:18:58.039 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.039 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.048 INFO:teuthology.orchestra.run.smithi032.stdout:5679 write 0xec93e6a thru 0xec97220 (0x33b7 bytes) 2024-09-13T09:18:58.053 INFO:teuthology.orchestra.run.smithi032.stdout:5680 read 0x12531ef thru 0x125dc63 (0xaa75 bytes) 2024-09-13T09:18:58.053 INFO:teuthology.orchestra.run.smithi032.stdout:5682 read 0x33b6c7d thru 0x33c142e (0xa7b2 bytes) 2024-09-13T09:18:58.053 INFO:teuthology.orchestra.run.smithi032.stdout:5685 write 0x11805f9 thru 0x11833ac (0x2db4 bytes) 2024-09-13T09:18:58.057 INFO:teuthology.orchestra.run.smithi032.stdout:5686 write 0xa1bd78c thru 0xa1bd9af (0x224 bytes) 2024-09-13T09:18:58.061 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.062 INFO:teuthology.orchestra.run.smithi032.stdout:5688 write 0x5618fce thru 0x561e0ae (0x50e1 bytes) 2024-09-13T09:18:58.065 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:58.062+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:58.085 INFO:teuthology.orchestra.run.smithi032.stdout:5689 trunc from 0xec97221 to 0x1f458c0 2024-09-13T09:18:58.121 INFO:teuthology.orchestra.run.smithi032.stdout:5690 write 0x4d3e99e thru 0x4d3fe5c (0x14bf bytes) 2024-09-13T09:18:58.125 INFO:teuthology.orchestra.run.smithi032.stdout:5691 writesame 0x46b29fc thru 0x46b2a00 data_size 0x1(0x5 bytes) 2024-09-13T09:18:58.131 INFO:teuthology.orchestra.run.smithi032.stdout:5692 write 0x2c30209 thru 0x2c325ff (0x23f7 bytes) 2024-09-13T09:18:58.134 INFO:teuthology.orchestra.run.smithi032.stdout:5693 write 0x182e080 thru 0x18323fc (0x437d bytes) 2024-09-13T09:18:58.140 INFO:teuthology.orchestra.run.smithi032.stdout:5694 read 0x13c139 thru 0x140196 (0x405e bytes) 2024-09-13T09:18:58.140 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.141 INFO:teuthology.orchestra.run.smithi032.stdout:5696 read 0x1e823c8 thru 0x1e90b69 (0xe7a2 bytes) 2024-09-13T09:18:58.141 INFO:teuthology.orchestra.run.smithi032.stdout:5697 punch from 0x3bc34d8 to 0x3bcfb1c, (0xc644 bytes) 2024-09-13T09:18:58.154 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:58.150+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:58.155 INFO:teuthology.orchestra.run.smithi032.stdout:5698 write 0x72222cc thru 0x7229a0b (0x7740 bytes) 2024-09-13T09:18:58.157 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:58.158 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:58.158 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:58.158 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:58.158 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:58.158 INFO:teuthology.orchestra.run.smithi032.stdout:5699 write 0x3555884 thru 0x355db47 (0x82c4 bytes) 2024-09-13T09:18:58.173 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:18:58.175 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:18:58.175 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:18:58.175 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T09:18:58.175 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.909676 2024-09-13T09:18:58.175 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0.9096761496617571 2024-09-13T09:18:58.181 INFO:teuthology.orchestra.run.smithi032.stdout:5700 writesame 0xb07c26e thru 0xb08768b data_size 0x6a(0xb41e bytes) 2024-09-13T09:18:58.181 INFO:teuthology.orchestra.run.smithi032.stdout:5701 write 0x15612c thru 0x1635b8 (0xd48d bytes) 2024-09-13T09:18:58.185 INFO:teuthology.orchestra.run.smithi032.stdout:5702 read 0x61d3fcf thru 0x61d4b68 (0xb9a bytes) 2024-09-13T09:18:58.185 INFO:teuthology.orchestra.run.smithi032.stdout:5703 read 0x7c1dffd thru 0x7c2a298 (0xc29c bytes) 2024-09-13T09:18:58.200 INFO:teuthology.orchestra.run.smithi032.stdout:5705 writesame 0xd865031 thru 0xd8691b5 data_size 0x1(0x4185 bytes) 2024-09-13T09:18:58.200 INFO:teuthology.orchestra.run.smithi032.stdout:5706 punch from 0x7e8fe8a to 0x7e9f873, (0xf9e9 bytes) 2024-09-13T09:18:58.201 INFO:teuthology.orchestra.run.smithi032.stdout:5707 writesame 0xca08dd3 thru 0xca101d5 data_size 0x1(0x7403 bytes) 2024-09-13T09:18:58.223 INFO:teuthology.orchestra.run.smithi032.stdout:5709 read 0x27ddd58 thru 0x27e27f5 (0x4a9e bytes) 2024-09-13T09:18:58.224 INFO:teuthology.orchestra.run.smithi032.stdout:5711 writesame 0x34f1087 thru 0x34f1c06 data_size 0x10(0xb80 bytes) 2024-09-13T09:18:58.240 INFO:teuthology.orchestra.run.smithi032.stdout:5713 writesame 0xe9f9f8f thru 0xea02b1f data_size 0x1(0x8b91 bytes) 2024-09-13T09:18:58.244 INFO:teuthology.orchestra.run.smithi032.stdout:5714 trunc from 0xea02b20 to 0xc8c576a 2024-09-13T09:18:58.255 INFO:teuthology.orchestra.run.smithi032.stdout:5715 write 0x165b61 thru 0x166818 (0xcb8 bytes) 2024-09-13T09:18:58.258 INFO:teuthology.orchestra.run.smithi032.stdout:5716 writesame 0xb38e895 thru 0xb39932c data_size 0x8(0xaa98 bytes) 2024-09-13T09:18:58.265 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:58.262+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:58.270 INFO:teuthology.orchestra.run.smithi032.stdout:5717 read 0x541e3de thru 0x5422145 (0x3d68 bytes) 2024-09-13T09:18:58.271 INFO:teuthology.orchestra.run.smithi032.stdout:5720 write 0xa37a537 thru 0xa385617 (0xb0e1 bytes) 2024-09-13T09:18:58.278 INFO:teuthology.orchestra.run.smithi032.stdout:5723 read 0xbeb1dce thru 0xbebd1a9 (0xb3dc bytes) 2024-09-13T09:18:58.278 INFO:teuthology.orchestra.run.smithi032.stdout:5724 read 0x98f750b thru 0x98ff5ff (0x80f5 bytes) 2024-09-13T09:18:58.278 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.278 INFO:teuthology.orchestra.run.smithi032.stdout:5726 read 0x533d1e0 thru 0x5348d76 (0xbb97 bytes) 2024-09-13T09:18:58.278 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.278 INFO:teuthology.orchestra.run.smithi032.stdout:5729 punch from 0x2ecd2cf to 0x2ed9812, (0xc543 bytes) 2024-09-13T09:18:58.278 INFO:teuthology.orchestra.run.smithi032.stdout:5730 read 0x4a45f11 thru 0x4a4b5bf (0x56af bytes) 2024-09-13T09:18:58.278 INFO:teuthology.orchestra.run.smithi032.stdout:5733 write 0xbb82a3e thru 0xbb8c58a (0x9b4d bytes) 2024-09-13T09:18:58.285 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.285 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.304 INFO:teuthology.orchestra.run.smithi032.stdout:5738 trunc from 0xc8c576a to 0x4147161 2024-09-13T09:18:58.327 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.327 INFO:teuthology.orchestra.run.smithi032.stdout:5740 punch from 0x2f2f41e to 0x2f38b39, (0x971b bytes) 2024-09-13T09:18:58.327 INFO:teuthology.orchestra.run.smithi032.stdout:5741 punch from 0x1382c90 to 0x13834ee, (0x85e bytes) 2024-09-13T09:18:58.327 INFO:teuthology.orchestra.run.smithi032.stdout:5742 read 0x364712f thru 0x36549b4 (0xd886 bytes) 2024-09-13T09:18:58.343 INFO:teuthology.orchestra.run.smithi032.stdout:5744 writesame 0x6311c03 thru 0x631bf8a data_size 0x4(0xa388 bytes) 2024-09-13T09:18:58.343 INFO:teuthology.orchestra.run.smithi032.stdout:5745 write 0x3ecf3a8 thru 0x3ed11e9 (0x1e42 bytes) 2024-09-13T09:18:58.345 INFO:teuthology.orchestra.run.smithi032.stdout:5748 read 0x183d810 thru 0x184d1e4 (0xf9d5 bytes) 2024-09-13T09:18:58.352 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:58.352 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:58.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.663501177000001, 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.21372158699999999, 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:58.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:18:58.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685363+0000", 2024-09-13T09:18:58.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:18:58.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685372+0000", 2024-09-13T09:18:58.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4590000000000004e-06 2024-09-13T09:18:58.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685394+0000", 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1616000000000001e-05 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685407+0000", 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3675e-05 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685484+0000", 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7065999999999997e-05 2024-09-13T09:18:58.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685493+0000", 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8070000000000006e-06 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690770+0000", 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052763869999999996 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690940+0000", 2024-09-13T09:18:58.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017001900000000001 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.691238+0000", 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029794499999999998 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.692057+0000", 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081943000000000001 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.898933+0000", 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.20687588200000001 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899026+0000", 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2777999999999998e-05 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899086+0000", 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0188e-05 2024-09-13T09:18:58.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.361 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.361 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:14590 3.1d 3:be60f898:::rbd_data.11853ecd5f5f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13672152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:18:58.361 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:58.361 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.807652736999998, 2024-09-13T09:18:58.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052557382, 2024-09-13T09:18:58.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:58.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 14590 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:18:58.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541207+0000", 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562477+0000", 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021270155999999998 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562539+0000", 2024-09-13T09:18:58.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1851000000000001e-05 2024-09-13T09:18:58.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562565+0000", 2024-09-13T09:18:58.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6194e-05 2024-09-13T09:18:58.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562702+0000", 2024-09-13T09:18:58.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013731299999999999 2024-09-13T09:18:58.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.563071+0000", 2024-09-13T09:18:58.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036892899999999999 2024-09-13T09:18:58.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:58.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.565167+0000", 2024-09-13T09:18:58.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020961959999999998 2024-09-13T09:18:58.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:58.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.585261+0000", 2024-09-13T09:18:58.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020093443999999999 2024-09-13T09:18:58.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:58.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593596+0000", 2024-09-13T09:18:58.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083352849999999996 2024-09-13T09:18:58.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:58.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593696+0000", 2024-09-13T09:18:58.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100021 2024-09-13T09:18:58.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593770+0000", 2024-09-13T09:18:58.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3889999999999999e-05 2024-09-13T09:18:58.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.367 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:14584 3.2 e39/22 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head v 39'64, mlcod=38'62)", 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.805667091000004, 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051546091000000002, 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:18:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543197+0000", 2024-09-13T09:18:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588691+0000", 2024-09-13T09:18:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045493475999999998 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588711+0000", 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0485000000000001e-05 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588721+0000", 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.499e-06 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588766+0000", 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5228000000000001e-05 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588788+0000", 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2503000000000001e-05 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594671+0000", 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058822809999999996 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594744+0000", 2024-09-13T09:18:58.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3776000000000007e-05 2024-09-13T09:18:58.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.372 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.372 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:22580 3.1c 3:39876c9f:::rbd_data.1185d6f8ec87.000000000000000f:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T09:18:58.372 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:58.372 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.496414301000002, 2024-09-13T09:18:58.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050372896, 2024-09-13T09:18:58.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:58.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22580 2024-09-13T09:18:58.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.373 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:58.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852449+0000", 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852462+0000", 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3756e-05 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852495+0000", 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2435999999999999e-05 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852516+0000", 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1347000000000001e-05 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852644+0000", 2024-09-13T09:18:58.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012777200000000001 2024-09-13T09:18:58.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852784+0000", 2024-09-13T09:18:58.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013980500000000001 2024-09-13T09:18:58.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:58.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.860936+0000", 2024-09-13T09:18:58.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081526859999999993 2024-09-13T09:18:58.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:58.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902522+0000", 2024-09-13T09:18:58.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041585203000000001 2024-09-13T09:18:58.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:58.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902560+0000", 2024-09-13T09:18:58.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8603000000000003e-05 2024-09-13T09:18:58.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:58.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902708+0000", 2024-09-13T09:18:58.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014740900000000001 2024-09-13T09:18:58.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902824+0000", 2024-09-13T09:18:58.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116519 2024-09-13T09:18:58.378 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54746 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8011776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.117483825000001, 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041426971999999999, 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54746 2024-09-13T09:18:58.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.380 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:58.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231382+0000", 2024-09-13T09:18:58.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231381+0000", 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231384+0000", 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.599e-06 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231390+0000", 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.231395+0000", 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0170000000000004e-06 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272716+0000", 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041321670999999997 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272767+0000", 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0447e-05 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.272809+0000", 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1659999999999998e-05 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54986 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6348800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.106575226, 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041375935000000003, 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54986 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242290+0000", 2024-09-13T09:18:58.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242292+0000", 2024-09-13T09:18:58.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3039999999999999e-06 2024-09-13T09:18:58.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242298+0000", 2024-09-13T09:18:58.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:18:58.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242303+0000", 2024-09-13T09:18:58.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8359999999999999e-06 2024-09-13T09:18:58.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283539+0000", 2024-09-13T09:18:58.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041235330000000001 2024-09-13T09:18:58.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283593+0000", 2024-09-13T09:18:58.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4926000000000002e-05 2024-09-13T09:18:58.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283666+0000", 2024-09-13T09:18:58.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2718000000000004e-05 2024-09-13T09:18:58.388 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.388 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54987 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6447104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.106558044, 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041604369000000002, 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54987 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242309+0000", 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1160000000000002e-06 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242317+0000", 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9799999999999998e-06 2024-09-13T09:18:58.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242323+0000", 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6439999999999997e-06 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283711+0000", 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388085999999998 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283766+0000", 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5112999999999999e-05 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283912+0000", 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014575300000000001 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54988 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6545408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.393 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.106538224000001, 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041865404000000002, 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54988 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:58.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242329+0000", 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242335+0000", 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242341+0000", 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9440000000000002e-06 2024-09-13T09:18:58.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283952+0000", 2024-09-13T09:18:58.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041610981999999998 2024-09-13T09:18:58.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284076+0000", 2024-09-13T09:18:58.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124356 2024-09-13T09:18:58.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284193+0000", 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116345 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54989 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6643712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.106520110000002, 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042123657000000002, 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54989 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242348+0000", 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.655e-06 2024-09-13T09:18:58.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242354+0000", 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2269999999999998e-06 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242360+0000", 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6170000000000003e-06 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284248+0000", 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041887830000000001 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284340+0000", 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1358000000000002e-05 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284469+0000", 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012932300000000001 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54990 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6742016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.106486820000001, 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042278333000000001, 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54990 2024-09-13T09:18:58.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242378+0000", 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242381+0000", 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0680000000000001e-06 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242392+0000", 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0762e-05 2024-09-13T09:18:58.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242398+0000", 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1040000000000003e-06 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284506+0000", 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042107983000000002 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284564+0000", 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.821e-05 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284657+0000", 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2588000000000002e-05 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54991 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6840320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.407 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.106462028999999, 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042489459, 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54991 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242406+0000", 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242412+0000", 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5919999999999997e-06 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242425+0000", 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2713e-05 2024-09-13T09:18:58.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284694+0000", 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042269558999999998 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284784+0000", 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9999000000000003e-05 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284893+0000", 2024-09-13T09:18:58.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108422 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54992 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6938624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.106435657999999, 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042636897999999999, 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54992 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:18:58.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242429+0000", 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242432+0000", 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7599999999999998e-06 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242439+0000", 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5760000000000002e-06 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242448+0000", 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284935+0000", 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042486949000000003 2024-09-13T09:18:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284991+0000", 2024-09-13T09:18:58.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6471000000000002e-05 2024-09-13T09:18:58.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285067+0000", 2024-09-13T09:18:58.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5543000000000005e-05 2024-09-13T09:18:58.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.416 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54993 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7036928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.106412781, 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042867959999999997, 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54993 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.417 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242452+0000", 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242455+0000", 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4360000000000001e-06 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242463+0000", 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8220000000000007e-06 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242468+0000", 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9479999999999999e-06 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285107+0000", 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042639415999999999 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285196+0000", 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8721000000000007e-05 2024-09-13T09:18:58.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285321+0000", 2024-09-13T09:18:58.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012495 2024-09-13T09:18:58.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.424 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54994 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7135232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.106385617000001, 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043009622999999997, 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54994 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:18:58.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242482+0000", 2024-09-13T09:18:58.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:18:58.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242486+0000", 2024-09-13T09:18:58.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1049999999999997e-06 2024-09-13T09:18:58.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242489+0000", 2024-09-13T09:18:58.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.004e-06 2024-09-13T09:18:58.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285354+0000", 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.04286479 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285413+0000", 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9778000000000003e-05 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285489+0000", 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6141000000000002e-05 2024-09-13T09:18:58.429 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.430 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.430 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54995 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7233536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.430 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:58.430 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.106374246000001, 2024-09-13T09:18:58.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043166641999999998, 2024-09-13T09:18:58.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54995 2024-09-13T09:18:58.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.431 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:58.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:58.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:18:58.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242492+0000", 2024-09-13T09:18:58.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.437e-06 2024-09-13T09:18:58.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242496+0000", 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242498+0000", 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5950000000000001e-06 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285528+0000", 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043029755000000003 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285585+0000", 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285658+0000", 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2713999999999995e-05 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.436 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54996 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7331840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.436 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:58.436 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.106364931000002, 2024-09-13T09:18:58.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043333782000000001, 2024-09-13T09:18:58.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54996 2024-09-13T09:18:58.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.436 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242500+0000", 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242502+0000", 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3599999999999999e-06 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242505+0000", 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1439999999999999e-06 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242507+0000", 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.605e-06 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285698+0000", 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043190911999999998 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285754+0000", 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5541999999999998e-05 2024-09-13T09:18:58.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285834+0000", 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0426000000000006e-05 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54997 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.106355841999999, 2024-09-13T09:18:58.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043496174999999998, 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54997 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242509+0000", 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242511+0000", 2024-09-13T09:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.313e-06 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242514+0000", 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242516+0000", 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3999999999999999e-06 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285875+0000", 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043358578000000002 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285929+0000", 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4602999999999999e-05 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286006+0000", 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6429999999999998e-05 2024-09-13T09:18:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54998 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7528448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.10626323, 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043666469999999999, 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.445 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.446 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54998 2024-09-13T09:18:58.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.446 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:58.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242518+0000", 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999161 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242606+0000", 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7607000000000003e-05 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242617+0000", 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0882e-05 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242624+0000", 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9009999999999997e-06 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286046+0000", 2024-09-13T09:18:58.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043422308 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286106+0000", 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0285000000000003e-05 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286269+0000", 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000162578 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:55006 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8314880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.102874562, 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041461771000000001, 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 55006 2024-09-13T09:18:58.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245993+0000", 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2869999999999999e-06 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245999+0000", 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0159999999999997e-06 2024-09-13T09:18:58.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.246004+0000", 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287345+0000", 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041341217 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287398+0000", 2024-09-13T09:18:58.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2840000000000002e-05 2024-09-13T09:18:58.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287453+0000", 2024-09-13T09:18:58.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4746000000000001e-05 2024-09-13T09:18:58.454 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.454 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.454 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.454 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:68662 3.1d 3:be8be2fa:::rbd_data.118556027ff4.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.191819121000002, 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.044923689000000003, 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 68662 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157045+0000", 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157052+0000", 2024-09-13T09:18:58.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8750000000000002e-06 2024-09-13T09:18:58.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157066+0000", 2024-09-13T09:18:58.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3815000000000001e-05 2024-09-13T09:18:58.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157073+0000", 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.64e-06 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157145+0000", 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2517999999999999e-05 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157267+0000", 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012185599999999999 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.164012+0000", 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0067447660000000001 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201805+0000", 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037793528999999999 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201881+0000", 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5678000000000003e-05 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201933+0000", 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2346999999999999e-05 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201970+0000", 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6532000000000002e-05 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.460 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.461 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:58.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:18:58.463 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:58.458+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:58.464 INFO:teuthology.orchestra.run.smithi032.stdout:5749 write 0xe9b563f thru 0xe9bb5a3 (0x5f65 bytes) 2024-09-13T09:18:58.464 INFO:teuthology.orchestra.run.smithi032.stdout:5752 write 0x6afcfb4 thru 0x6b0adc0 (0xde0d bytes) 2024-09-13T09:18:58.464 INFO:teuthology.orchestra.run.smithi032.stdout:5753 read 0x461e565 thru 0x461f2df (0xd7b bytes) 2024-09-13T09:18:58.464 INFO:teuthology.orchestra.run.smithi032.stdout:5754 read 0x199a894 thru 0x199b58a (0xcf7 bytes) 2024-09-13T09:18:58.464 INFO:teuthology.orchestra.run.smithi032.stdout:5755 write 0xc3cbee8 thru 0xc3db058 (0xf171 bytes) 2024-09-13T09:18:58.464 INFO:teuthology.orchestra.run.smithi032.stdout:5757 trunc from 0xe9bb5a4 to 0x860c5e3 2024-09-13T09:18:58.464 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.464 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.464 INFO:teuthology.orchestra.run.smithi032.stdout:5761 punch from 0x1969c63 to 0x196ce3a, (0x31d7 bytes) 2024-09-13T09:18:58.465 INFO:teuthology.orchestra.run.smithi032.stdout:5762 trunc from 0x860c5e3 to 0xc32b4af 2024-09-13T09:18:58.465 INFO:teuthology.orchestra.run.smithi032.stdout:5763 read 0x8f75e05 thru 0x8f7ed9c (0x8f98 bytes) 2024-09-13T09:18:58.465 INFO:teuthology.orchestra.run.smithi032.stdout:5764 read 0x65718b5 thru 0x657a481 (0x8bcd bytes) 2024-09-13T09:18:58.465 INFO:teuthology.orchestra.run.smithi032.stdout:5769 read 0x7205ebe thru 0x720d02e (0x7171 bytes) 2024-09-13T09:18:58.465 INFO:teuthology.orchestra.run.smithi032.stdout:5770 write 0xdb2f28d thru 0xdb3c022 (0xcd96 bytes) 2024-09-13T09:18:58.465 INFO:teuthology.orchestra.run.smithi032.stdout:5771 write 0x8bde57f thru 0x8be25a6 (0x4028 bytes) 2024-09-13T09:18:58.470 INFO:teuthology.orchestra.run.smithi032.stdout:5772 writesame 0x11e0235 thru 0x11e22d2 data_size 0x19(0x209e bytes) 2024-09-13T09:18:58.474 INFO:teuthology.orchestra.run.smithi032.stdout:5773 read 0x915fba0 thru 0x916cfd3 (0xd434 bytes) 2024-09-13T09:18:58.474 INFO:teuthology.orchestra.run.smithi032.stdout:5774 write 0xda3a6fa thru 0xda42965 (0x826c bytes) 2024-09-13T09:18:58.478 INFO:teuthology.orchestra.run.smithi032.stdout:5775 write 0xa3496e5 thru 0xa34c88e (0x31aa bytes) 2024-09-13T09:18:58.486 INFO:teuthology.orchestra.run.smithi032.stdout:5776 write 0x1fa59d3 thru 0x1fab9aa (0x5fd8 bytes) 2024-09-13T09:18:58.493 INFO:teuthology.orchestra.run.smithi032.stdout:5777 read 0x2940e81 thru 0x294f8dc (0xea5c bytes) 2024-09-13T09:18:58.493 INFO:teuthology.orchestra.run.smithi032.stdout:5779 read 0xac0b4a9 thru 0xac0eb95 (0x36ed bytes) 2024-09-13T09:18:58.493 INFO:teuthology.orchestra.run.smithi032.stdout:5780 read 0x892d4a0 thru 0x8930334 (0x2e95 bytes) 2024-09-13T09:18:58.495 INFO:teuthology.orchestra.run.smithi032.stdout:5781 punch from 0x56f077d to 0x56fbe05, (0xb688 bytes) 2024-09-13T09:18:58.495 INFO:teuthology.orchestra.run.smithi032.stdout:5782 read 0x85072d7 thru 0x8507f28 (0xc52 bytes) 2024-09-13T09:18:58.495 INFO:teuthology.orchestra.run.smithi032.stdout:5783 read 0x2a639eb thru 0x2a69590 (0x5ba6 bytes) 2024-09-13T09:18:58.495 INFO:teuthology.orchestra.run.smithi032.stdout:5784 read 0x20af92c thru 0x20b1943 (0x2018 bytes) 2024-09-13T09:18:58.495 INFO:teuthology.orchestra.run.smithi032.stdout:5785 punch from 0x1513a39 to 0x151c508, (0x8acf bytes) 2024-09-13T09:18:58.518 INFO:teuthology.orchestra.run.smithi032.stdout:5786 trunc from 0xdb3c023 to 0x354568a 2024-09-13T09:18:58.542 INFO:teuthology.orchestra.run.smithi032.stdout:5787 read 0x1e488a thru 0x1ee274 (0x99eb bytes) 2024-09-13T09:18:58.563 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:58.558+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:58.572 INFO:teuthology.orchestra.run.smithi032.stdout:5788 writesame 0x9b6d5f2 thru 0x9b77bad data_size 0x2(0xa5bc bytes) 2024-09-13T09:18:58.572 INFO:teuthology.orchestra.run.smithi032.stdout:5789 write 0x2e09dbd thru 0x2e12dfd (0x9041 bytes) 2024-09-13T09:18:58.576 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.576 INFO:teuthology.orchestra.run.smithi032.stdout:5792 write 0x1221eb4 thru 0x122eadf (0xcc2c bytes) 2024-09-13T09:18:58.586 INFO:teuthology.orchestra.run.smithi032.stdout:5793 trunc from 0x9b77bae to 0x6ee4b31 2024-09-13T09:18:58.596 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.6 primary-affinity to 0.909676 (8596162) 2024-09-13T09:18:58.601 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.601 INFO:teuthology.orchestra.run.smithi032.stdout:5796 punch from 0x3e7aef9 to 0x3e8a5e3, (0xf6ea bytes) 2024-09-13T09:18:58.619 INFO:teuthology.orchestra.run.smithi032.stdout:5797 write 0xa334006 thru 0xa33a14d (0x6148 bytes) 2024-09-13T09:18:58.643 INFO:teuthology.orchestra.run.smithi032.stdout:5798 write 0xe0cad41 thru 0xe0d3150 (0x8410 bytes) 2024-09-13T09:18:58.646 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:58.647 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:58.647 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:58.647 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:58.650 INFO:teuthology.orchestra.run.smithi032.stdout:5799 write 0x539c4e thru 0x540745 (0x6af8 bytes) 2024-09-13T09:18:58.657 INFO:teuthology.orchestra.run.smithi032.stdout:5800 writesame 0x453a58a thru 0x454849d data_size 0x6(0xdf14 bytes) 2024-09-13T09:18:58.661 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:18:58.664 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:58.661+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:58.670 INFO:teuthology.orchestra.run.smithi032.stdout:5801 read 0x9c8082b thru 0x9c86add (0x62b3 bytes) 2024-09-13T09:18:58.670 INFO:teuthology.orchestra.run.smithi032.stdout:5802 read 0x97d014c thru 0x97dd87f (0xd734 bytes) 2024-09-13T09:18:58.670 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.685 INFO:teuthology.orchestra.run.smithi032.stdout:5804 trunc from 0xe0d3151 to 0x63917c3 2024-09-13T09:18:58.708 INFO:teuthology.orchestra.run.smithi032.stdout:5806 writesame 0xf68a44 thru 0xf6b39b data_size 0x38(0x2958 bytes) 2024-09-13T09:18:58.715 INFO:teuthology.orchestra.run.smithi032.stdout:5809 write 0x535a6b5 thru 0x5363943 (0x928f bytes) 2024-09-13T09:18:58.720 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.720 INFO:teuthology.orchestra.run.smithi032.stdout:5813 write 0xb84416 thru 0xb8d20b (0x8df6 bytes) 2024-09-13T09:18:58.723 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.729 INFO:teuthology.orchestra.run.smithi032.stdout:5818 trunc from 0x63917c3 to 0x88c0f3f 2024-09-13T09:18:58.740 INFO:teuthology.orchestra.run.smithi032.stdout:5819 punch from 0x1e52b8 to 0x1ed85e, (0x85a6 bytes) 2024-09-13T09:18:58.741 INFO:teuthology.orchestra.run.smithi032.stdout:5822 punch from 0x5a0c1b5 to 0x5a10dfe, (0x4c49 bytes) 2024-09-13T09:18:58.741 INFO:teuthology.orchestra.run.smithi032.stdout:5823 read 0x43dcd11 thru 0x43eb3b7 (0xe6a7 bytes) 2024-09-13T09:18:58.741 INFO:teuthology.orchestra.run.smithi032.stdout:5824 read 0x29e9677 thru 0x29f9544 (0xfece bytes) 2024-09-13T09:18:58.742 INFO:teuthology.orchestra.run.smithi032.stdout:5825 read 0x3df1976 thru 0x3df2ad7 (0x1162 bytes) 2024-09-13T09:18:58.747 INFO:teuthology.orchestra.run.smithi032.stdout:5826 trunc from 0x88c0f3f to 0xa4c61db 2024-09-13T09:18:58.759 INFO:teuthology.orchestra.run.smithi032.stdout:5827 write 0x83d3f6e thru 0x83d5d4f (0x1de2 bytes) 2024-09-13T09:18:58.764 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:18:58.761+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:18:58.765 INFO:teuthology.orchestra.run.smithi032.stdout:5828 punch from 0x7d006d6 to 0x7d059bf, (0x52e9 bytes) 2024-09-13T09:18:58.765 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:58.786 INFO:teuthology.orchestra.run.smithi032.stdout:5832 writesame 0xd9eea0b thru 0xd9f8a1c data_size 0x7(0xa012 bytes) 2024-09-13T09:18:58.787 INFO:teuthology.orchestra.run.smithi032.stdout:5835 write 0xa11e0b8 thru 0xa12a71e (0xc667 bytes) 2024-09-13T09:18:58.794 INFO:teuthology.orchestra.run.smithi032.stdout:5837 read 0x63a7a72 thru 0x63ad194 (0x5723 bytes) 2024-09-13T09:18:58.795 INFO:teuthology.orchestra.run.smithi032.stdout:5838 punch from 0x32048ef to 0x320f232, (0xa943 bytes) 2024-09-13T09:18:58.795 INFO:teuthology.orchestra.run.smithi032.stdout:5839 read 0x2211d88 thru 0x221b526 (0x979f bytes) 2024-09-13T09:18:58.795 INFO:teuthology.orchestra.run.smithi032.stdout:5841 read 0xa622ee2 thru 0xa62c4c6 (0x95e5 bytes) 2024-09-13T09:18:58.827 INFO:teuthology.orchestra.run.smithi032.stdout:5842 trunc from 0xd9f8a1d to 0x1e41e0e 2024-09-13T09:18:58.836 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:58.836 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:58.836 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:58.836 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:58.836 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:58.854 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:18:58.881 INFO:teuthology.orchestra.run.smithi032.stdout:5843 write 0x4fb8488 thru 0x4fc5d81 (0xd8fa bytes) 2024-09-13T09:18:58.887 INFO:teuthology.orchestra.run.smithi032.stdout:5844 read 0x1fa5d74 thru 0x1fa777d (0x1a0a bytes) 2024-09-13T09:18:58.897 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:18:58.894+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:18:58.908 INFO:teuthology.orchestra.run.smithi032.stdout:5845 write 0x7d7c405 thru 0x7d8acd9 (0xe8d5 bytes) 2024-09-13T09:18:58.913 INFO:teuthology.orchestra.run.smithi032.stdout:5846 read 0x75afebe thru 0x75b5f3a (0x607d bytes) 2024-09-13T09:18:58.913 INFO:teuthology.orchestra.run.smithi032.stdout:5847 punch from 0x3452ec6 to 0x345dfe5, (0xb11f bytes) 2024-09-13T09:18:58.914 INFO:teuthology.orchestra.run.smithi032.stdout:5848 read 0x16a036d thru 0x16afdd7 (0xfa6b bytes) 2024-09-13T09:18:58.930 INFO:teuthology.orchestra.run.smithi032.stdout:5849 trunc from 0x7d8acda to 0x1c7758f 2024-09-13T09:18:58.966 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:58.961+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:58.978 INFO:teuthology.orchestra.run.smithi032.stdout:5851 write 0x5c936f7 thru 0x5c94a0e (0x1318 bytes) 2024-09-13T09:18:58.986 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:50082 3.10 3:0ec43ecb:::rbd_object_map.1185172eb410:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T09:18:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.619607921, 2024-09-13T09:18:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80943783999999996, 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 50082 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362619+0000", 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362632+0000", 2024-09-13T09:18:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3338e-05 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362657+0000", 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5207000000000001e-05 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362669+0000", 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1768e-05 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362738+0000", 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9486999999999994e-05 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362748+0000", 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0010000000000001e-05 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169473+0000", 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80672482300000004 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169884+0000", 2024-09-13T09:18:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041114299999999999 2024-09-13T09:18:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.170206+0000", 2024-09-13T09:18:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032127700000000002 2024-09-13T09:18:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171827+0000", 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016217880000000001 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171871+0000", 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171967+0000", 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5752000000000001e-05 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.172059+0000", 2024-09-13T09:18:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2082000000000004e-05 2024-09-13T09:18:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56772 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 4112384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.551919802, 2024-09-13T09:18:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054477114, 2024-09-13T09:18:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56772 2024-09-13T09:18:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430309+0000", 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430312+0000", 2024-09-13T09:18:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.255e-06 2024-09-13T09:18:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430319+0000", 2024-09-13T09:18:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4760000000000001e-06 2024-09-13T09:18:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430324+0000", 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9799999999999998e-06 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484634+0000", 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054310031000000002 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484697+0000", 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3355999999999994e-05 2024-09-13T09:18:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484787+0000", 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9350000000000003e-05 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56785 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5390336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548136320000001, 2024-09-13T09:18:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054655007999999998, 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56785 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434094+0000", 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.471e-06 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434097+0000", 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0740000000000001e-06 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434100+0000", 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7829999999999999e-06 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488422+0000", 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054321855000000002 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488563+0000", 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014154 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488748+0000", 2024-09-13T09:18:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018461399999999999 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56786 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548126937999999, 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054950539999999999, 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56786 2024-09-13T09:18:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:18:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434104+0000", 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4020000000000001e-06 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434107+0000", 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.072e-06 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434109+0000", 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.717e-06 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488795+0000", 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054685661000000003 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488869+0000", 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4265999999999997e-05 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489053+0000", 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018374299999999999 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56787 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5586944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548116679, 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055115725999999997, 2024-09-13T09:18:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56787 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434112+0000", 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434114+0000", 2024-09-13T09:18:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.308e-06 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434117+0000", 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434119+0000", 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.588e-06 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489097+0000", 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054977824000000002 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489146+0000", 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8503e-05 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489228+0000", 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2694000000000005e-05 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56788 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548107787999999, 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055437562000000003, 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56788 2024-09-13T09:18:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434121+0000", 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434123+0000", 2024-09-13T09:18:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2190000000000001e-06 2024-09-13T09:18:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434126+0000", 2024-09-13T09:18:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:18:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434128+0000", 2024-09-13T09:18:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8250000000000001e-06 2024-09-13T09:18:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489293+0000", 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055164092999999997 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489428+0000", 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013572599999999999 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489559+0000", 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013086 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56789 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548098638999999, 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055674077000000002, 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56789 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434132+0000", 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.302e-06 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434136+0000", 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434139+0000", 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489583+0000", 2024-09-13T09:18:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055443994000000003 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489699+0000", 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116611 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489805+0000", 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105331 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56790 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548088264, 2024-09-13T09:18:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055716839999999997, 2024-09-13T09:18:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56790 2024-09-13T09:18:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:18:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434142+0000", 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2559999999999999e-06 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434145+0000", 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.96e-06 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434148+0000", 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6479999999999999e-06 2024-09-13T09:18:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489851+0000", 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055702792000000001 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489858+0000", 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3900000000000004e-06 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56791 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548079491999999, 2024-09-13T09:18:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055740029000000003, 2024-09-13T09:18:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56791 2024-09-13T09:18:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:18:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434151+0000", 2024-09-13T09:18:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:18:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434154+0000", 2024-09-13T09:18:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0139999999999999e-06 2024-09-13T09:18:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434157+0000", 2024-09-13T09:18:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6699999999999998e-06 2024-09-13T09:18:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489884+0000", 2024-09-13T09:18:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055727511 2024-09-13T09:18:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489890+0000", 2024-09-13T09:18:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7200000000000003e-06 2024-09-13T09:18:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56792 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548070563, 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055760363, 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56792 2024-09-13T09:18:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:18:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434163+0000", 2024-09-13T09:18:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-13T09:18:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434167+0000", 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.292e-06 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434169+0000", 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489914+0000", 2024-09-13T09:18:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055744415999999998 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489919+0000", 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56793 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548057043, 2024-09-13T09:18:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055774049999999999, 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56793 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434173+0000", 2024-09-13T09:18:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-06 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434177+0000", 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1070000000000001e-06 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434179+0000", 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.666e-06 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489941+0000", 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055762071000000003 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489946+0000", 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56794 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548047935, 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055791704999999997, 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56794 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434182+0000", 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1629999999999999e-06 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434185+0000", 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9950000000000001e-06 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434188+0000", 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489968+0000", 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055779789000000003 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489973+0000", 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2800000000000003e-06 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56795 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548038998999999, 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055809612000000001, 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56795 2024-09-13T09:18:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434191+0000", 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.212e-06 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434194+0000", 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.078e-06 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434198+0000", 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9820000000000002e-06 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489995+0000", 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055796363000000002 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490000+0000", 2024-09-13T09:18:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1839999999999998e-06 2024-09-13T09:18:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56796 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548028737999999, 2024-09-13T09:18:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055825859999999998, 2024-09-13T09:18:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56796 2024-09-13T09:18:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434200+0000", 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434202+0000", 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.204e-06 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434205+0000", 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9689999999999999e-06 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434207+0000", 2024-09-13T09:18:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490021+0000", 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055813929999999998 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490027+0000", 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.259e-06 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56797 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.548020018999999, 2024-09-13T09:18:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055843649000000002, 2024-09-13T09:18:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56797 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:18:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434210+0000", 2024-09-13T09:18:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2389999999999999e-06 2024-09-13T09:18:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434213+0000", 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434216+0000", 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7800000000000001e-06 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490048+0000", 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055831981000000003 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490053+0000", 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8640000000000004e-06 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56807 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.546339861, 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054407720999999999, 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56807 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435889+0000", 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435891+0000", 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.217e-06 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435894+0000", 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.067e-06 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435896+0000", 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490292+0000", 2024-09-13T09:18:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054395840000000001 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490297+0000", 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56808 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.546331035000001, 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05442524, 2024-09-13T09:18:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56808 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435900+0000", 2024-09-13T09:18:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:18:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435903+0000", 2024-09-13T09:18:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:18:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435905+0000", 2024-09-13T09:18:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:18:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490319+0000", 2024-09-13T09:18:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054413396000000003 2024-09-13T09:18:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490324+0000", 2024-09-13T09:18:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:18:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56809 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.546320979000001, 2024-09-13T09:18:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054442786, 2024-09-13T09:18:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56809 2024-09-13T09:18:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:18:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:18:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435910+0000", 2024-09-13T09:18:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:18:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435913+0000", 2024-09-13T09:18:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1470000000000001e-06 2024-09-13T09:18:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435915+0000", 2024-09-13T09:18:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6769999999999999e-06 2024-09-13T09:18:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490346+0000", 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054430574000000002 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490351+0000", 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2700000000000004e-06 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56810 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.546311679999999, 2024-09-13T09:18:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054459855000000001, 2024-09-13T09:18:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56810 2024-09-13T09:18:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:18:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435917+0000", 2024-09-13T09:18:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435919+0000", 2024-09-13T09:18:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.457e-06 2024-09-13T09:18:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435924+0000", 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435929+0000", 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8300000000000003e-06 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490373+0000", 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054443841999999999 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490378+0000", 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8509999999999998e-06 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56811 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.546282579, 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054456683999999998, 2024-09-13T09:18:59.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56811 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:18:59.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435946+0000", 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435949+0000", 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435956+0000", 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8719999999999996e-06 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435962+0000", 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000004e-06 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490398+0000", 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054436824000000002 2024-09-13T09:18:59.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490403+0000", 2024-09-13T09:18:59.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0710000000000002e-06 2024-09-13T09:18:59.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.081 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.081 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.081 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:59.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:18:59.084 INFO:teuthology.orchestra.run.smithi032.stdout:5852 write 0xc49002b thru 0xc498d33 (0x8d09 bytes) 2024-09-13T09:18:59.084 INFO:teuthology.orchestra.run.smithi032.stdout:5853 trunc from 0xc498d34 to 0xcd4332c 2024-09-13T09:18:59.084 INFO:teuthology.orchestra.run.smithi032.stdout:5854 read 0x7d2f570 thru 0x7d379f1 (0x8482 bytes) 2024-09-13T09:18:59.084 INFO:teuthology.orchestra.run.smithi032.stdout:5855 write 0xebefd49 thru 0xebf2ad3 (0x2d8b bytes) 2024-09-13T09:18:59.084 INFO:teuthology.orchestra.run.smithi032.stdout:5856 punch from 0x78d6f1a to 0x78d9a1b, (0x2b01 bytes) 2024-09-13T09:18:59.084 INFO:teuthology.orchestra.run.smithi032.stdout:5857 write 0x5aefa90 thru 0x5af4f3a (0x54ab bytes) 2024-09-13T09:18:59.084 INFO:teuthology.orchestra.run.smithi032.stdout:5858 writesame 0xace5e41 thru 0xaceb9e6 data_size 0x1(0x5ba6 bytes) 2024-09-13T09:18:59.084 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:59.077+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:59.106 INFO:teuthology.orchestra.run.smithi032.stdout:5859 write 0x35c95f8 thru 0x35d31d9 (0x9be2 bytes) 2024-09-13T09:18:59.110 INFO:teuthology.orchestra.run.smithi032.stdout:5861 write 0x943157c thru 0x9436905 (0x538a bytes) 2024-09-13T09:18:59.118 INFO:teuthology.orchestra.run.smithi032.stdout:5862 punch from 0x9313af2 to 0x9322fb8, (0xf4c6 bytes) 2024-09-13T09:18:59.119 INFO:teuthology.orchestra.run.smithi032.stdout:5863 writesame 0xb7518ed thru 0xb75634a data_size 0x39(0x4a5e bytes) 2024-09-13T09:18:59.127 INFO:teuthology.orchestra.run.smithi032.stdout:5865 read 0x68a9268 thru 0x68ab3c4 (0x215d bytes) 2024-09-13T09:18:59.150 INFO:teuthology.orchestra.run.smithi032.stdout:5866 trunc from 0xebf2ad4 to 0x66748fa 2024-09-13T09:18:59.177 INFO:teuthology.orchestra.run.smithi032.stdout:5867 read 0x5cced29 thru 0x5cdeae9 (0xfdc1 bytes) 2024-09-13T09:18:59.184 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:18:59.181+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:18:59.211 INFO:teuthology.orchestra.run.smithi032.stdout:5868 writesame 0xd7a7fb8 thru 0xd7b7a7b data_size 0x16(0xfac4 bytes) 2024-09-13T09:18:59.211 INFO:teuthology.orchestra.run.smithi032.stdout:5869 writesame 0xb9b9130 thru 0xb9c1ac1 data_size 0x1(0x8992 bytes) 2024-09-13T09:18:59.257 INFO:teuthology.orchestra.run.smithi032.stdout:5870 write 0x722fcdb thru 0x723e7de (0xeb04 bytes) 2024-09-13T09:18:59.265 INFO:teuthology.orchestra.run.smithi032.stdout:5873 writesame 0xbfbed01 thru 0xbfc4d78 data_size 0x1(0x6078 bytes) 2024-09-13T09:18:59.270 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:59.270 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:59.270 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:18:59.270 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:59.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:18:59.285 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:59.278+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:59.386 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:59.381+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:59.452 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:59.452 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:18:59.452 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:18:59.453 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:18:59.453 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:59.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:18:59.514 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:18:59.510+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:18:59.587 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:18:59.588 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:18:59.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:18:59.588 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:18:59.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.588 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:18:59.588 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:18:59.588 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.19296785, 2024-09-13T09:18:59.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077478195999999999, 2024-09-13T09:18:59.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.588 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:59.588 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.588 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:18:59.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392636+0000", 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392651+0000", 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392674+0000", 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.264e-05 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392691+0000", 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7203000000000001e-05 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392780+0000", 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8671999999999994e-05 2024-09-13T09:18:59.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392794+0000", 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.466664+0000", 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073869913999999995 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467139+0000", 2024-09-13T09:18:59.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047488599999999999 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467664+0000", 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052530600000000002 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469125+0000", 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001460432 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469847+0000", 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072197799999999996 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470006+0000", 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015955900000000001 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470116+0000", 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109836 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:49389 3.d 3:b0b9272f:::rbd_data.1185e728d948.0000000000000025:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.502649073999997, 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077194925999999997, 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 49389 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.595 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082955+0000", 2024-09-13T09:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082967+0000", 2024-09-13T09:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2609000000000001e-05 2024-09-13T09:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082998+0000", 2024-09-13T09:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0596000000000003e-05 2024-09-13T09:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083013+0000", 2024-09-13T09:18:59.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4766999999999999e-05 2024-09-13T09:18:59.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083136+0000", 2024-09-13T09:18:59.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012295599999999999 2024-09-13T09:18:59.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083360+0000", 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000224661 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.097720+0000", 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014359627 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.159975+0000", 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062254728000000002 2024-09-13T09:18:59.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.159999+0000", 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.427e-05 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.160089+0000", 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9895999999999994e-05 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.160152+0000", 2024-09-13T09:18:59.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2939000000000002e-05 2024-09-13T09:18:59.601 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.601 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.601 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.601 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52425 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6111232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.601 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805609+0000", 2024-09-13T09:18:59.601 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.779996449999999, 2024-09-13T09:18:59.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064134332000000002, 2024-09-13T09:18:59.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.602 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.602 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.602 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52425 2024-09-13T09:18:59.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.602 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805609+0000", 2024-09-13T09:18:59.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805609+0000", 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805609+0000", 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805612+0000", 2024-09-13T09:18:59.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.27e-06 2024-09-13T09:18:59.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805620+0000", 2024-09-13T09:18:59.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:18:59.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805627+0000", 2024-09-13T09:18:59.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8669999999999996e-06 2024-09-13T09:18:59.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869445+0000", 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063818099000000003 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869583+0000", 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138078 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869744+0000", 2024-09-13T09:18:59.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160462 2024-09-13T09:18:59.606 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.606 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.606 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.606 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52426 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6209536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.606 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:18:59.606 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.779972793999999, 2024-09-13T09:18:59.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064408950000000006, 2024-09-13T09:18:59.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52426 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:18:59.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805632+0000", 2024-09-13T09:18:59.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805645+0000", 2024-09-13T09:18:59.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2469999999999999e-05 2024-09-13T09:18:59.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805654+0000", 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7760000000000003e-06 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805661+0000", 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.114e-06 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869801+0000", 2024-09-13T09:18:59.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064139777999999995 2024-09-13T09:18:59.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869918+0000", 2024-09-13T09:18:59.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117199 2024-09-13T09:18:59.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870042+0000", 2024-09-13T09:18:59.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124196 2024-09-13T09:18:59.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52427 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6307840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.779938864000002, 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064720860000000005, 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52427 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805666+0000", 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805670+0000", 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2200000000000001e-06 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805678+0000", 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.32e-06 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805685+0000", 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.883e-06 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870103+0000", 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064417962999999995 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870241+0000", 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138362 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870388+0000", 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146732 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52428 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.779913134000001, 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064990774000000001, 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52428 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805692+0000", 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805695+0000", 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805704+0000", 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5299999999999996e-06 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805712+0000", 2024-09-13T09:18:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870433+0000", 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064720343999999999 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870544+0000", 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011108900000000001 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870683+0000", 2024-09-13T09:18:59.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000139705 2024-09-13T09:18:59.620 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.620 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.620 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.620 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52429 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6504448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.620 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:18:59.620 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.779872837999999, 2024-09-13T09:18:59.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065238731999999994, 2024-09-13T09:18:59.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52429 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805732+0000", 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805736+0000", 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8600000000000003e-06 2024-09-13T09:18:59.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805746+0000", 2024-09-13T09:18:59.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6860000000000001e-06 2024-09-13T09:18:59.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805753+0000", 2024-09-13T09:18:59.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:18:59.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870724+0000", 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064970450999999999 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870836+0000", 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112796 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870972+0000", 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135382 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52430 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6602752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.779846612, 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065499180000000004, 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.625 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52430 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:18:59.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805762+0000", 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805773+0000", 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1496e-05 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805780+0000", 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871018+0000", 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065237396000000003 2024-09-13T09:18:59.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871124+0000", 2024-09-13T09:18:59.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106469 2024-09-13T09:18:59.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871258+0000", 2024-09-13T09:18:59.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134175 2024-09-13T09:18:59.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.629 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52431 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6701056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.779819757999999, 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065755623999999999, 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52431 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.630 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805785+0000", 2024-09-13T09:18:59.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805789+0000", 2024-09-13T09:18:59.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2160000000000002e-06 2024-09-13T09:18:59.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805797+0000", 2024-09-13T09:18:59.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4870000000000002e-06 2024-09-13T09:18:59.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805804+0000", 2024-09-13T09:18:59.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:18:59.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871298+0000", 2024-09-13T09:18:59.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065494259999999999 2024-09-13T09:18:59.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871405+0000", 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106512 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871542+0000", 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013683599999999999 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.635 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52432 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6799360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.635 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:18:59.635 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.779755058999999, 2024-09-13T09:18:59.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065976444999999995, 2024-09-13T09:18:59.635 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.635 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.635 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.635 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.635 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.635 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52432 2024-09-13T09:18:59.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.635 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:18:59.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:18:59.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805850+0000", 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805855+0000", 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8670000000000002e-06 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805867+0000", 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2597e-05 2024-09-13T09:18:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805876+0000", 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.833e-06 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871584+0000", 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065707765000000001 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871690+0000", 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105832 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871827+0000", 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013744500000000001 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52440 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7585792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:18:59.639 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.775631246, 2024-09-13T09:18:59.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064276712999999999, 2024-09-13T09:18:59.640 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.640 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.640 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.640 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.640 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.640 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52440 2024-09-13T09:18:59.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.640 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:18:59.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:18:59.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809974+0000", 2024-09-13T09:18:59.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809976+0000", 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8920000000000001e-06 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809981+0000", 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5170000000000001e-06 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809985+0000", 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9539999999999998e-06 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.873936+0000", 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063951470999999996 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874076+0000", 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014022199999999999 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874251+0000", 2024-09-13T09:18:59.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017497500000000001 2024-09-13T09:18:59.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.644 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.644 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52441 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7684096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.644 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:18:59.644 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.775617867000001, 2024-09-13T09:18:59.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064567507999999996, 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52441 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809989+0000", 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8589999999999999e-06 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809994+0000", 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809999+0000", 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874317+0000", 2024-09-13T09:18:59.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064318742999999998 2024-09-13T09:18:59.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874423+0000", 2024-09-13T09:18:59.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106116 2024-09-13T09:18:59.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874555+0000", 2024-09-13T09:18:59.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013196700000000001 2024-09-13T09:18:59.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.649 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.649 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52442 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7782400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.649 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:18:59.649 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.775603864000001, 2024-09-13T09:18:59.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064804082999999998, 2024-09-13T09:18:59.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.649 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52442 2024-09-13T09:18:59.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.650 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:18:59.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810003+0000", 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8360000000000001e-06 2024-09-13T09:18:59.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810009+0000", 2024-09-13T09:18:59.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0220000000000001e-06 2024-09-13T09:18:59.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810013+0000", 2024-09-13T09:18:59.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9879999999999998e-06 2024-09-13T09:18:59.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874593+0000", 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064579851999999993 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874688+0000", 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4581999999999997e-05 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874806+0000", 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118108 2024-09-13T09:18:59.653 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.654 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.654 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52443 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7880704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.654 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:18:59.654 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.775588968000001, 2024-09-13T09:18:59.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065175650000000002, 2024-09-13T09:18:59.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.654 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.654 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.654 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52443 2024-09-13T09:18:59.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.655 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:18:59.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:18:59.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:18:59.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810018+0000", 2024-09-13T09:18:59.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.798e-06 2024-09-13T09:18:59.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810023+0000", 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7829999999999998e-06 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810027+0000", 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8890000000000004e-06 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874886+0000", 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064859362000000004 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875029+0000", 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014311900000000001 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875192+0000", 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000163012 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.658 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52444 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7979008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.775575389, 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065471830999999994, 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52444 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.659 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:18:59.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810032+0000", 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8199999999999999e-06 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810036+0000", 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5510000000000001e-06 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810041+0000", 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0679999999999998e-06 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875256+0000", 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065215765999999994 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875366+0000", 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109639 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875502+0000", 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013630399999999999 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52445 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.775562073, 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065733503999999998, 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52445 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810043+0000", 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810045+0000", 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.813e-06 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810050+0000", 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5940000000000004e-06 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810054+0000", 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439999999999998e-06 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875543+0000", 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065489393000000007 2024-09-13T09:18:59.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875658+0000", 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115302 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875777+0000", 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118776 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52446 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8175616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.775548743999998, 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065969204000000004, 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52446 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:18:59.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810059+0000", 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.84e-06 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810063+0000", 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4329999999999997e-06 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810067+0000", 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8800000000000001e-06 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875813+0000", 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065745910000000005 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875903+0000", 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0665e-05 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876026+0000", 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122836 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52447 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8273920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.775514503, 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066176867, 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52447 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810093+0000", 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6089999999999999e-06 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810099+0000", 2024-09-13T09:18:59.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1539999999999999e-06 2024-09-13T09:18:59.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810104+0000", 2024-09-13T09:18:59.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4630000000000003e-06 2024-09-13T09:18:59.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876062+0000", 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065957672999999994 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876153+0000", 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1119000000000006e-05 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876268+0000", 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115431 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.676 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52448 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.775498335000002, 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066534826000000005, 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52448 2024-09-13T09:18:59.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810109+0000", 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0169999999999999e-06 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810115+0000", 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810120+0000", 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9620000000000001e-06 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876349+0000", 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066229213999999995 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876484+0000", 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013503199999999999 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876642+0000", 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158228 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52452 3.d 3:b5f40919:::rbd_data.118567e41306.0000000000000055:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.775482496999999, 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070307575999999997, 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:18:59.681 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52452 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810126+0000", 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810132+0000", 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7349999999999999e-06 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810136+0000", 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2069999999999997e-06 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876306+0000", 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066169943999999994 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876520+0000", 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021377500000000001 2024-09-13T09:18:59.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.877399+0000", 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087836900000000005 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880158+0000", 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002759657 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:18:59.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880212+0000", 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3551999999999998e-05 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880315+0000", 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010306099999999999 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880431+0000", 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011603300000000001 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.686 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:18:59.687 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:18:59.687 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:18:59.687 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:18:59.582+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:18:59.688 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:18:59.688 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:18:59.690 INFO:teuthology.orchestra.run.smithi032.stdout:5874 read 0xa00d78b thru 0xa018b77 (0xb3ed bytes) 2024-09-13T09:18:59.690 INFO:teuthology.orchestra.run.smithi032.stdout:5876 read 0xad2db30 thru 0xad36578 (0x8a49 bytes) 2024-09-13T09:18:59.691 INFO:teuthology.orchestra.run.smithi032.stdout:5878 read 0x41910cc thru 0x4196a71 (0x59a6 bytes) 2024-09-13T09:18:59.691 INFO:teuthology.orchestra.run.smithi032.stdout:5879 punch from 0x97ae266 to 0x97bb0b5, (0xce4f bytes) 2024-09-13T09:18:59.691 INFO:teuthology.orchestra.run.smithi032.stdout:5880 writesame 0x7155517 thru 0x715b0ab data_size 0xf(0x5b95 bytes) 2024-09-13T09:18:59.691 INFO:teuthology.orchestra.run.smithi032.stdout:5881 read 0xccf394a thru 0xccf8b55 (0x520c bytes) 2024-09-13T09:18:59.691 INFO:teuthology.orchestra.run.smithi032.stdout:5883 trunc from 0xd7b7a7c to 0x409ef58 2024-09-13T09:18:59.691 INFO:teuthology.orchestra.run.smithi032.stdout:5884 write 0x232778a thru 0x232d73c (0x5fb3 bytes) 2024-09-13T09:18:59.691 INFO:teuthology.orchestra.run.smithi032.stdout:5886 read 0x283cbff thru 0x284a6f4 (0xdaf6 bytes) 2024-09-13T09:18:59.691 INFO:teuthology.orchestra.run.smithi032.stdout:5887 punch from 0xf2f23d to 0xf3bc02, (0xc9c5 bytes) 2024-09-13T09:18:59.691 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:59.685+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:18:59.719 INFO:teuthology.orchestra.run.smithi032.stdout:5888 writesame 0xba1ad6a thru 0xba27245 data_size 0x56(0xc4dc bytes) 2024-09-13T09:18:59.719 INFO:teuthology.orchestra.run.smithi032.stdout:5889 read 0x2cbe28c thru 0x2cc95b2 (0xb327 bytes) 2024-09-13T09:18:59.720 INFO:teuthology.orchestra.run.smithi032.stdout:5890 punch from 0x29177 to 0x2fae0, (0x6969 bytes) 2024-09-13T09:18:59.720 INFO:teuthology.orchestra.run.smithi032.stdout:5891 write 0x616b8ea thru 0x6170b21 (0x5238 bytes) 2024-09-13T09:18:59.727 INFO:teuthology.orchestra.run.smithi032.stdout:5892 writesame 0x53e904c thru 0x53f5308 data_size 0x1(0xc2bd bytes) 2024-09-13T09:18:59.791 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:18:59.786+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:18:59.796 INFO:teuthology.orchestra.run.smithi032.stdout:5893 trunc from 0xba27246 to 0xd10602e 2024-09-13T09:18:59.810 INFO:teuthology.orchestra.run.smithi032.stdout:5894 punch from 0x83f2756 to 0x83f89ca, (0x6274 bytes) 2024-09-13T09:18:59.810 INFO:teuthology.orchestra.run.smithi032.stdout:5896 punch from 0x448d8ce to 0x448efd5, (0x1707 bytes) 2024-09-13T09:18:59.810 INFO:teuthology.orchestra.run.smithi032.stdout:5897 read 0x1f463c7 thru 0x1f551fe (0xee38 bytes) 2024-09-13T09:18:59.811 INFO:teuthology.orchestra.run.smithi032.stdout:5898 writesame 0x91907da thru 0x919b5ed data_size 0xd(0xae14 bytes) 2024-09-13T09:18:59.820 INFO:teuthology.orchestra.run.smithi032.stdout:5899 read 0x49870f4 thru 0x49878ae (0x7bb bytes) 2024-09-13T09:18:59.820 INFO:teuthology.orchestra.run.smithi032.stdout:5900 write 0x4378f72 thru 0x438760b (0xe69a bytes) 2024-09-13T09:18:59.824 INFO:teuthology.orchestra.run.smithi032.stdout:5901 writesame 0x9c88530 thru 0x9c95068 data_size 0x5(0xcb39 bytes) 2024-09-13T09:18:59.834 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:59.834 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:59.834 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:18:59.834 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:59.841 INFO:teuthology.orchestra.run.smithi032.stdout:5903 read 0x28aa0e9 thru 0x28b22bd (0x81d5 bytes) 2024-09-13T09:18:59.841 INFO:teuthology.orchestra.run.smithi032.stdout:5904 read 0xc535f81 thru 0xc53c18a (0x620a bytes) 2024-09-13T09:18:59.841 INFO:teuthology.orchestra.run.smithi032.stdout:5905 writesame 0x10bd644 thru 0x10c2c32 data_size 0x1(0x55ef bytes) 2024-09-13T09:18:59.851 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:18:59.866 INFO:teuthology.orchestra.run.smithi032.stdout:5907 read 0xcef2aa4 thru 0xcef5b36 (0x3093 bytes) 2024-09-13T09:18:59.866 INFO:teuthology.orchestra.run.smithi032.stdout:5908 write 0x81b0bd2 thru 0x81bfb10 (0xef3f bytes) 2024-09-13T09:18:59.873 INFO:teuthology.orchestra.run.smithi032.stdout:5909 read 0x974b52e thru 0x974d3d8 (0x1eab bytes) 2024-09-13T09:18:59.873 INFO:teuthology.orchestra.run.smithi032.stdout:5910 write 0x6c21357 thru 0x6c2fb8b (0xe835 bytes) 2024-09-13T09:18:59.880 INFO:teuthology.orchestra.run.smithi032.stdout:5912 write 0x49bde4d thru 0x49c6fc1 (0x9175 bytes) 2024-09-13T09:18:59.883 INFO:teuthology.orchestra.run.smithi032.stdout:5915 read 0x9dc9a36 thru 0x9dd1973 (0x7f3e bytes) 2024-09-13T09:18:59.884 INFO:teuthology.orchestra.run.smithi032.stdout:5917 read 0x487cbbd thru 0x487e4de (0x1922 bytes) 2024-09-13T09:18:59.891 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:18:59.889+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:18:59.897 INFO:teuthology.orchestra.run.smithi032.stdout:5918 writesame 0xde1376e thru 0xde1e2bd data_size 0x8(0xab50 bytes) 2024-09-13T09:18:59.897 INFO:teuthology.orchestra.run.smithi032.stdout:5919 punch from 0x11e36d2 to 0x11eb6d2, (0x8000 bytes) 2024-09-13T09:18:59.897 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:18:59.897 INFO:teuthology.orchestra.run.smithi032.stdout:5921 read 0x80d5f5f thru 0x80d9e49 (0x3eeb bytes) 2024-09-13T09:18:59.898 INFO:teuthology.orchestra.run.smithi032.stdout:5923 punch from 0x8352a17 to 0x835b6ec, (0x8cd5 bytes) 2024-09-13T09:18:59.898 INFO:teuthology.orchestra.run.smithi032.stdout:5924 read 0xc559374 thru 0xc56921d (0xfeaa bytes) 2024-09-13T09:18:59.898 INFO:teuthology.orchestra.run.smithi032.stdout:5925 read 0xbd8484e thru 0xbd9411c (0xf8cf bytes) 2024-09-13T09:18:59.898 INFO:teuthology.orchestra.run.smithi032.stdout:5926 read 0x4af7b78 thru 0x4b00713 (0x8b9c bytes) 2024-09-13T09:18:59.898 INFO:teuthology.orchestra.run.smithi032.stdout:5927 read 0x10f64e1 thru 0x10fa1b1 (0x3cd1 bytes) 2024-09-13T09:18:59.906 INFO:teuthology.orchestra.run.smithi032.stdout:5928 trunc from 0xde1e2be to 0x978da94 2024-09-13T09:18:59.926 INFO:teuthology.orchestra.run.smithi032.stdout:5929 writesame 0x2d6ea12 thru 0x2d7238d data_size 0x1a(0x397c bytes) 2024-09-13T09:18:59.940 INFO:teuthology.orchestra.run.smithi032.stdout:5930 trunc from 0x978da94 to 0xdb49dc2 2024-09-13T09:18:59.952 INFO:teuthology.orchestra.run.smithi032.stdout:5931 writesame 0x5bdc7ed thru 0x5bec1cb data_size 0x1(0xf9df bytes) 2024-09-13T09:18:59.987 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:18:59.987 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:18:59.987 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:18:59.987 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:18:59.987 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:18:59.992 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:18:59.989+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:00.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:19:00.024 INFO:teuthology.orchestra.run.smithi032.stdout:5932 punch from 0x978ee45 to 0x9792aa2, (0x3c5d bytes) 2024-09-13T09:19:00.024 INFO:teuthology.orchestra.run.smithi032.stdout:5933 write 0x7d566c5 thru 0x7d615b3 (0xaeef bytes) 2024-09-13T09:19:00.027 INFO:teuthology.orchestra.run.smithi032.stdout:5934 write 0x778bc1b thru 0x779bae0 (0xfec6 bytes) 2024-09-13T09:19:00.030 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:19:00.031 INFO:teuthology.orchestra.run.smithi032.stdout:5936 read 0x265a864 thru 0x2669a2a (0xf1c7 bytes) 2024-09-13T09:19:00.031 INFO:teuthology.orchestra.run.smithi032.stdout:5937 read 0x5b74337 thru 0x5b819d0 (0xd69a bytes) 2024-09-13T09:19:00.032 INFO:teuthology.orchestra.run.smithi032.stdout:5939 read 0x1b23060 thru 0x1b23d31 (0xcd2 bytes) 2024-09-13T09:19:00.033 INFO:teuthology.orchestra.run.smithi032.stdout:5940 punch from 0x7a06b9b to 0x7a0e978, (0x7ddd bytes) 2024-09-13T09:19:00.033 INFO:teuthology.orchestra.run.smithi032.stdout:5944 punch from 0x53268e4 to 0x5331b83, (0xb29f bytes) 2024-09-13T09:19:00.033 INFO:teuthology.orchestra.run.smithi032.stdout:5945 write 0xdab910a thru 0xdabc9f4 (0x38eb bytes) 2024-09-13T09:19:00.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:19:00.039 INFO:teuthology.orchestra.run.smithi032.stdout:5947 read 0x59213b4 thru 0x592af45 (0x9b92 bytes) 2024-09-13T09:19:00.039 INFO:teuthology.orchestra.run.smithi032.stdout:5948 read 0x2f1f453 thru 0x2f22400 (0x2fae bytes) 2024-09-13T09:19:00.040 INFO:teuthology.orchestra.run.smithi032.stdout:5950 write 0x1e7d547 thru 0x1e8aca5 (0xd75f bytes) 2024-09-13T09:19:00.044 INFO:teuthology.orchestra.run.smithi032.stdout:5952 read 0x868ad4f thru 0x8698ece (0xe180 bytes) 2024-09-13T09:19:00.045 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:19:00.045 INFO:teuthology.orchestra.run.smithi032.stdout:5954 read 0x724ebba thru 0x724fb5e (0xfa5 bytes) 2024-09-13T09:19:00.045 INFO:teuthology.orchestra.run.smithi032.stdout:5955 read 0x14950a7 thru 0x14a4206 (0xf160 bytes) 2024-09-13T09:19:00.045 INFO:teuthology.orchestra.run.smithi032.stdout:5956 read 0xa3b28e1 thru 0xa3b837f (0x5a9f bytes) 2024-09-13T09:19:00.045 INFO:teuthology.orchestra.run.smithi032.stdout:5957 punch from 0xc280c50 to 0xc289c86, (0x9036 bytes) 2024-09-13T09:19:00.064 INFO:teuthology.orchestra.run.smithi032.stdout:5958 trunc from 0xdb49dc2 to 0xbe4382 2024-09-13T09:19:00.092 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:00.089+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:5959 write 0x619733 thru 0x61d013 (0x38e1 bytes) 2024-09-13T09:19:00.122 INFO:teuthology.orchestra.run.smithi032.stdout:5962 write 0x50a8098 thru 0x50adcef (0x5c58 bytes) 2024-09-13T09:19:00.127 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:19:00.127 INFO:teuthology.orchestra.run.smithi032.stdout:5965 read 0x23ec29 thru 0x23fc51 (0x1029 bytes) 2024-09-13T09:19:00.134 INFO:teuthology.orchestra.run.smithi032.stdout:5966 trunc from 0x50adcf0 to 0xe98be4 2024-09-13T09:19:00.156 INFO:teuthology.orchestra.run.smithi032.stdout:5967 read 0x9a2851 thru 0x9b0365 (0xdb15 bytes) 2024-09-13T09:19:00.157 INFO:teuthology.orchestra.run.smithi032.stdout:5968 read 0xe566b0 thru 0xe6146e (0xadbf bytes) 2024-09-13T09:19:00.159 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:00.159 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:00.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:00.160 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:00.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.160 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3:ed699f0f:::rbd_data.118591b29fad.000000000000008c:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:19:00.160 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:00.160 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 77.295870738000005, 2024-09-13T09:19:00.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94878089700000001, 2024-09-13T09:19:00.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.160 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.160 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.160 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.160 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:19:00.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:19:00.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:19:00.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:19:00.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:19:00.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:19:00.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:00.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:19:00.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:19:00.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810654+0000", 2024-09-13T09:19:00.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94834837599999999 2024-09-13T09:19:00.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810755+0000", 2024-09-13T09:19:00.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100987 2024-09-13T09:19:00.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810828+0000", 2024-09-13T09:19:00.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2751999999999997e-05 2024-09-13T09:19:00.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.165 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19465 3.10 3:0e3a5b30:::rbd_data.1185b72576e4.0000000000000062:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 63.210026001000003, 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97489159000000003, 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19465 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947890+0000", 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:00.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947899+0000", 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6840000000000008e-06 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947917+0000", 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7731e-05 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947925+0000", 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4189999999999996e-06 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947990+0000", 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5909999999999997e-05 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947998+0000", 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4449999999999999e-06 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.919994+0000", 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97199595900000002 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920258+0000", 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026439500000000001 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920629+0000", 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037096299999999998 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.921797+0000", 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001167721 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922544+0000", 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00074661200000000001 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922628+0000", 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4016000000000003e-05 2024-09-13T09:19:00.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922783+0000", 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015585499999999999 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51771 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.413289528, 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083783107999999995, 2024-09-13T09:19:00.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51771 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:00.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:00.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744631+0000", 2024-09-13T09:19:00.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.943e-06 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744639+0000", 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744646+0000", 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5860000000000001e-06 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828151+0000", 2024-09-13T09:19:00.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083505732999999999 2024-09-13T09:19:00.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828286+0000", 2024-09-13T09:19:00.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013457200000000001 2024-09-13T09:19:00.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828411+0000", 2024-09-13T09:19:00.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012564999999999999 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51772 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.413266432, 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083939185999999999, 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.177 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51772 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:00.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744654+0000", 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2440000000000002e-06 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744662+0000", 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3659999999999992e-06 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744669+0000", 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6529999999999997e-06 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828455+0000", 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083785707000000001 2024-09-13T09:19:00.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828512+0000", 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7549000000000001e-05 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828591+0000", 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8274000000000003e-05 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51773 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.413242776000001, 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084139096999999996, 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51773 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.182 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744674+0000", 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744677+0000", 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9399999999999998e-06 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744689+0000", 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.153e-05 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744705+0000", 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5679e-05 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828658+0000", 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083953190999999996 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828734+0000", 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5784999999999994e-05 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828814+0000", 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579000000000002e-05 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51774 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.413205414, 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084301713, 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51774 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:00.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744723+0000", 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0832000000000001e-05 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744733+0000", 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031e-05 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744740+0000", 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0500000000000003e-06 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828867+0000", 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084127417999999995 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828932+0000", 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4202000000000006e-05 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829014+0000", 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51775 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.413171741999999, 2024-09-13T09:19:00.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084465871999999997, 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51775 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744749+0000", 2024-09-13T09:19:00.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5200000000000002e-06 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744757+0000", 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1100000000000003e-06 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744764+0000", 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8430000000000004e-06 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829058+0000", 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084293899000000005 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829111+0000", 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3315e-05 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829212+0000", 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010079200000000001 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.194 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51776 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.413148029999999, 2024-09-13T09:19:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084631102999999999, 2024-09-13T09:19:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51776 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744769+0000", 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744772+0000", 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.14e-06 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744781+0000", 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.276e-06 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744788+0000", 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0269999999999998e-06 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829254+0000", 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084466325999999994 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829310+0000", 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5852e-05 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829401+0000", 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1075000000000005e-05 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51777 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.413092762000002, 2024-09-13T09:19:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084901272, 2024-09-13T09:19:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51777 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744824+0000", 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744828+0000", 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744840+0000", 2024-09-13T09:19:00.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1997000000000001e-05 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744847+0000", 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0659999999999998e-06 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829460+0000", 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084613387999999998 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829555+0000", 2024-09-13T09:19:00.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4999000000000003e-05 2024-09-13T09:19:00.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829726+0000", 2024-09-13T09:19:00.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017100899999999999 2024-09-13T09:19:00.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.204 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51778 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.413040318, 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085026334999999995, 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51778 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744877+0000", 2024-09-13T09:19:00.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744881+0000", 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5999999999999998e-06 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744892+0000", 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1732999999999999e-05 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744899+0000", 2024-09-13T09:19:00.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0759999999999997e-06 2024-09-13T09:19:00.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829772+0000", 2024-09-13T09:19:00.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084872633000000003 2024-09-13T09:19:00.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829826+0000", 2024-09-13T09:19:00.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4330000000000003e-05 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829904+0000", 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7558999999999994e-05 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51779 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:00.209 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.413011748999999, 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085200608999999997, 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51779 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:00.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744905+0000", 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744909+0000", 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3500000000000001e-06 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744917+0000", 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3129999999999999e-06 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744924+0000", 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6059999999999999e-06 2024-09-13T09:19:00.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829976+0000", 2024-09-13T09:19:00.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085051775999999996 2024-09-13T09:19:00.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830030+0000", 2024-09-13T09:19:00.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4404000000000003e-05 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830107+0000", 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6796000000000001e-05 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51780 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:00.214 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.412988156000001, 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085367775000000007, 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51780 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:00.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744929+0000", 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744932+0000", 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744940+0000", 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1130000000000001e-06 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744947+0000", 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:19:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830148+0000", 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085201099000000002 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830205+0000", 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6623999999999999e-05 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830297+0000", 2024-09-13T09:19:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2693000000000002e-05 2024-09-13T09:19:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51781 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.412965119999999, 2024-09-13T09:19:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085544856000000002, 2024-09-13T09:19:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51781 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744952+0000", 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744955+0000", 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0960000000000001e-06 2024-09-13T09:19:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745048+0000", 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2806999999999994e-05 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745058+0000", 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0132999999999999e-05 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830338+0000", 2024-09-13T09:19:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085279761999999995 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830420+0000", 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1551000000000006e-05 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830498+0000", 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8103999999999996e-05 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51782 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1794048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.412853198000001, 2024-09-13T09:19:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085605889000000004, 2024-09-13T09:19:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51782 2024-09-13T09:19:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745064+0000", 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745068+0000", 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.523e-06 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745076+0000", 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.354e-06 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745091+0000", 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5274999999999999e-05 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830538+0000", 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085447238999999994 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830591+0000", 2024-09-13T09:19:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2481000000000001e-05 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830671+0000", 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9716999999999996e-05 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51783 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1892352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.412820551999999, 2024-09-13T09:19:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085732662000000001, 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51783 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745100+0000", 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745108+0000", 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4360000000000002e-06 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745115+0000", 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9269999999999999e-06 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830707+0000", 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085591133999999999 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830758+0000", 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1180000000000001e-05 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830830+0000", 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2268999999999995e-05 2024-09-13T09:19:00.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51784 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.412788584000001, 2024-09-13T09:19:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085902834999999997, 2024-09-13T09:19:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.233 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51784 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:00.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745132+0000", 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.36e-06 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745143+0000", 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745150+0000", 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0400000000000004e-06 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830870+0000", 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085720086000000001 2024-09-13T09:19:00.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830955+0000", 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5321000000000005e-05 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831032+0000", 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6811999999999996e-05 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51785 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.412233689000001, 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085542517999999998, 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51785 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.238 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745160+0000", 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994755 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745697+0000", 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00053713000000000003 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745731+0000", 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3767999999999999e-05 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745754+0000", 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3748999999999999e-05 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831075+0000", 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085321067 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831130+0000", 2024-09-13T09:19:00.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4156999999999997e-05 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831227+0000", 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7093e-05 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51789 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.409540932999999, 2024-09-13T09:19:00.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083633549000000001, 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51789 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748376+0000", 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748380+0000", 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7529999999999999e-06 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748392+0000", 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2376e-05 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748399+0000", 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0199999999999997e-06 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831866+0000", 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083466636999999996 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831933+0000", 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7144000000000006e-05 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832010+0000", 2024-09-13T09:19:00.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7352000000000001e-05 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51790 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.409512367000001, 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083812485000000006, 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.247 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51790 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748405+0000", 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748408+0000", 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3859999999999999e-06 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748417+0000", 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:19:00.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748424+0000", 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832055+0000", 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083631469 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832129+0000", 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4240999999999996e-05 2024-09-13T09:19:00.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832218+0000", 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8562999999999999e-05 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51791 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2678784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.409487947999999, 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083959632000000006, 2024-09-13T09:19:00.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51791 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748429+0000", 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748432+0000", 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0800000000000002e-06 2024-09-13T09:19:00.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748441+0000", 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9120000000000004e-06 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748470+0000", 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569e-05 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832260+0000", 2024-09-13T09:19:00.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083790586 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832317+0000", 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6054999999999998e-05 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832390+0000", 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3033000000000004e-05 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:00.256 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:00.256 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:00.257 INFO:teuthology.orchestra.run.smithi032.stdout:5969 trunc from 0xe98be4 to 0x7eb0ece 2024-09-13T09:19:00.257 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:19:00.257 INFO:teuthology.orchestra.run.smithi032.stdout:5971 punch from 0x194b3fd to 0x194f485, (0x4088 bytes) 2024-09-13T09:19:00.258 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:19:00.259 INFO:teuthology.orchestra.run.smithi032.stdout:5972 trunc from 0x7eb0ece to 0xc6631e0 2024-09-13T09:19:00.259 INFO:teuthology.orchestra.run.smithi032.stdout:5973 write 0x38eec8c thru 0x38f22a7 (0x361c bytes) 2024-09-13T09:19:00.259 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:00.259 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:00.260 INFO:teuthology.orchestra.run.smithi032.stdout:5974 read 0xba97e4 thru 0xbb889b (0xf0b8 bytes) 2024-09-13T09:19:00.260 INFO:teuthology.orchestra.run.smithi032.stdout:5975 writesame 0xd0a1233 thru 0xd0a2e15 data_size 0xb(0x1be3 bytes) 2024-09-13T09:19:00.260 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:19:00.260 INFO:teuthology.orchestra.run.smithi032.stdout:5978 punch from 0x5ed8f8f to 0x5edf423, (0x6494 bytes) 2024-09-13T09:19:00.260 INFO:teuthology.orchestra.run.smithi032.stdout:5979 write 0x16c1d thru 0x20f7c (0xa360 bytes) 2024-09-13T09:19:00.260 INFO:teuthology.orchestra.run.smithi032.stdout:5980 write 0xc2b39b0 thru 0xc2c2012 (0xe663 bytes) 2024-09-13T09:19:00.261 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:00.261 INFO:teuthology.orchestra.run.smithi032.stdout:5981 writesame 0x73d4593 thru 0x73e19ad data_size 0x1(0xd41b bytes) 2024-09-13T09:19:00.261 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:00.262 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:00.250+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:00.262 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:00.275 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:00.276 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:00.294 INFO:teuthology.orchestra.run.smithi032.stdout:5982 write 0xbacde87 thru 0xbad828b (0xa405 bytes) 2024-09-13T09:19:00.299 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:00.299 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:00.300 INFO:teuthology.orchestra.run.smithi032.stdout:5983 read 0x2c4a06f thru 0x2c4d6ac (0x363e bytes) 2024-09-13T09:19:00.301 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:19:00.301 INFO:teuthology.orchestra.run.smithi032.stdout:5986 punch from 0x130d8cb to 0x131a740, (0xce75 bytes) 2024-09-13T09:19:00.301 INFO:teuthology.orchestra.run.smithi032.stdout:5987 writesame 0x2e9821b thru 0x2e98335 data_size 0x1(0x11b bytes) 2024-09-13T09:19:00.304 INFO:teuthology.orchestra.run.smithi032.stdout:5988 writesame 0x47fbe18 thru 0x4808c38 data_size 0x1(0xce21 bytes) 2024-09-13T09:19:00.323 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:00.323 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:00.345 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:00.357 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:19:00.357 INFO:teuthology.orchestra.run.smithi032.stdout:5991 read 0x547fcf8 thru 0x548948d (0x9796 bytes) 2024-09-13T09:19:00.358 INFO:teuthology.orchestra.run.smithi032.stdout:5992 write 0x7386e9c thru 0x738e120 (0x7285 bytes) 2024-09-13T09:19:00.358 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:00.353+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:00.372 INFO:teuthology.orchestra.run.smithi032.stdout:5993 writesame 0xd1d183b thru 0xd1e05c0 data_size 0x1(0xed86 bytes) 2024-09-13T09:19:00.372 INFO:teuthology.orchestra.run.smithi032.stdout:5994 write 0xb0f2834 thru 0xb101caa (0xf477 bytes) 2024-09-13T09:19:00.377 INFO:teuthology.orchestra.run.smithi032.stdout:5995 punch from 0x299f228 to 0x29a83a6, (0x917e bytes) 2024-09-13T09:19:00.377 INFO:teuthology.orchestra.run.smithi032.stdout:5996 read 0xbc51022 thru 0xbc5c07a (0xb059 bytes) 2024-09-13T09:19:00.377 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-13T09:19:00.378 INFO:teuthology.orchestra.run.smithi032.stdout:5998 writesame 0x8115303 thru 0x8118520 data_size 0x5(0x321e bytes) 2024-09-13T09:19:00.385 INFO:teuthology.orchestra.run.smithi032.stdout:6000 writesame 0x9fa991b thru 0x9fb626e data_size 0x1e(0xc954 bytes) 2024-09-13T09:19:00.400 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:00.401 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:00.409 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-13T09:19:00.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:19:00.459 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:00.454+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:00.561 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:00.554+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:00.591 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:00.591 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:00.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:19:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:19:00.685 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:00.681+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:00.729 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:00.729 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:00.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:19:00.783 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:00.781+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:00.864 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:00.861+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:00.904 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:00.904 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:00.918 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:19:01.007 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:01.005+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:01.066 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:01.066+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:01.080 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:01.080 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:01.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:19:01.167 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:01.162+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:01.266 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:01.267 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:01.267 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:01.262+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:01.281 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:19:01.368 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:01.366+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:01.409 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:01.409 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:01.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:19:01.469 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:01.462+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:01.538 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:01.539 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:01.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:19:01.599 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:01.597+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:01.671 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:01.669+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:01.682 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:01.682 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:01.698 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:19:01.772 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:01.766+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:01.839 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:01.840 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:01.854 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:19:01.902 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:01.898+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:01.973 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:01.970+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:02.033 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:02.033 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:02.047 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:19:02.090 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:02.086+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:02.176 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:02.173+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:02.211 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:02.212 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:02.226 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:19:02.277 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:02.273+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:02.353 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:02.354 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:02.364 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:19:02.368 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:19:02.414 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:02.410+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:02.479 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:02.473+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:02.491 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:02.491 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:02.506 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:19:02.524 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:02.540 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:02.541 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:02.556 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:02.557 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:02.573 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:02.573 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:02.579 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:02.577+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:02.592 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:02.592 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:02.610 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:02.610 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:02.628 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:02.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:19:02.658 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:02.659 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:02.681 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:02.674+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:02.693 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:02.693 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:02.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:02.693 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:02.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.694 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:19:02.694 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:19:02.694 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 94.004030817, 2024-09-13T09:19:02.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.21372158699999999, 2024-09-13T09:19:02.694 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:02.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10 2024-09-13T09:19:02.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.694 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:19:02.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:19:02.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685363+0000", 2024-09-13T09:19:02.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:19:02.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685372+0000", 2024-09-13T09:19:02.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4590000000000004e-06 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685394+0000", 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1616000000000001e-05 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685407+0000", 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3675e-05 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685484+0000", 2024-09-13T09:19:02.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7065999999999997e-05 2024-09-13T09:19:02.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:19:02.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685493+0000", 2024-09-13T09:19:02.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8070000000000006e-06 2024-09-13T09:19:02.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690770+0000", 2024-09-13T09:19:02.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052763869999999996 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690940+0000", 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017001900000000001 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.691238+0000", 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029794499999999998 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.692057+0000", 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081943000000000001 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.898933+0000", 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.20687588200000001 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899026+0000", 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2777999999999998e-05 2024-09-13T09:19:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899086+0000", 2024-09-13T09:19:02.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0188e-05 2024-09-13T09:19:02.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.702 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.702 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.702 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:14590 3.1d 3:be60f898:::rbd_data.11853ecd5f5f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13672152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:19:02.702 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:19:02.702 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.148182376999998, 2024-09-13T09:19:02.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052557382, 2024-09-13T09:19:02.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.702 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 14590 2024-09-13T09:19:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:19:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:19:02.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541207+0000", 2024-09-13T09:19:02.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-13T09:19:02.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562477+0000", 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021270155999999998 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562539+0000", 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1851000000000001e-05 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562565+0000", 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6194e-05 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562702+0000", 2024-09-13T09:19:02.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013731299999999999 2024-09-13T09:19:02.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.563071+0000", 2024-09-13T09:19:02.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036892899999999999 2024-09-13T09:19:02.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:02.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.565167+0000", 2024-09-13T09:19:02.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020961959999999998 2024-09-13T09:19:02.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:02.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.585261+0000", 2024-09-13T09:19:02.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020093443999999999 2024-09-13T09:19:02.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:02.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593596+0000", 2024-09-13T09:19:02.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083352849999999996 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593696+0000", 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100021 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593770+0000", 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3889999999999999e-05 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:14584 3.2 e39/22 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head v 39'64, mlcod=38'62)", 2024-09-13T09:19:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:19:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.146196731000003, 2024-09-13T09:19:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051546091000000002, 2024-09-13T09:19:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:19:02.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:19:02.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543197+0000", 2024-09-13T09:19:02.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:02.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588691+0000", 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045493475999999998 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588711+0000", 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0485000000000001e-05 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588721+0000", 2024-09-13T09:19:02.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.499e-06 2024-09-13T09:19:02.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588766+0000", 2024-09-13T09:19:02.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5228000000000001e-05 2024-09-13T09:19:02.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588788+0000", 2024-09-13T09:19:02.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2503000000000001e-05 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594671+0000", 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058822809999999996 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594744+0000", 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3776000000000007e-05 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.713 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:22580 3.1c 3:39876c9f:::rbd_data.1185d6f8ec87.000000000000000f:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.836943941000001, 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050372896, 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22580 2024-09-13T09:19:02.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852449+0000", 2024-09-13T09:19:02.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:02.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852462+0000", 2024-09-13T09:19:02.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3756e-05 2024-09-13T09:19:02.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852495+0000", 2024-09-13T09:19:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2435999999999999e-05 2024-09-13T09:19:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852516+0000", 2024-09-13T09:19:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1347000000000001e-05 2024-09-13T09:19:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852644+0000", 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012777200000000001 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852784+0000", 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013980500000000001 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.860936+0000", 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081526859999999993 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902522+0000", 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041585203000000001 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902560+0000", 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8603000000000003e-05 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902708+0000", 2024-09-13T09:19:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014740900000000001 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902824+0000", 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116519 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54987 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6447104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:19:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.447087684, 2024-09-13T09:19:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041604369000000002, 2024-09-13T09:19:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54987 2024-09-13T09:19:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:19:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:19:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:19:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242309+0000", 2024-09-13T09:19:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1160000000000002e-06 2024-09-13T09:19:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242317+0000", 2024-09-13T09:19:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9799999999999998e-06 2024-09-13T09:19:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242323+0000", 2024-09-13T09:19:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6439999999999997e-06 2024-09-13T09:19:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283711+0000", 2024-09-13T09:19:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388085999999998 2024-09-13T09:19:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283766+0000", 2024-09-13T09:19:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5112999999999999e-05 2024-09-13T09:19:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283912+0000", 2024-09-13T09:19:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014575300000000001 2024-09-13T09:19:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54988 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6545408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:19:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.447067864000001, 2024-09-13T09:19:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041865404000000002, 2024-09-13T09:19:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54988 2024-09-13T09:19:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:19:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:19:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:19:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242329+0000", 2024-09-13T09:19:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:19:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242335+0000", 2024-09-13T09:19:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:19:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242341+0000", 2024-09-13T09:19:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9440000000000002e-06 2024-09-13T09:19:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283952+0000", 2024-09-13T09:19:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041610981999999998 2024-09-13T09:19:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284076+0000", 2024-09-13T09:19:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124356 2024-09-13T09:19:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284193+0000", 2024-09-13T09:19:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116345 2024-09-13T09:19:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54989 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6643712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:19:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.447049750000001, 2024-09-13T09:19:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042123657000000002, 2024-09-13T09:19:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54989 2024-09-13T09:19:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:19:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242348+0000", 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.655e-06 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242354+0000", 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2269999999999998e-06 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242360+0000", 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6170000000000003e-06 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284248+0000", 2024-09-13T09:19:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041887830000000001 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284340+0000", 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1358000000000002e-05 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284469+0000", 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012932300000000001 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54990 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6742016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:19:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.44701646, 2024-09-13T09:19:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042278333000000001, 2024-09-13T09:19:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54990 2024-09-13T09:19:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:19:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:19:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242378+0000", 2024-09-13T09:19:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242381+0000", 2024-09-13T09:19:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0680000000000001e-06 2024-09-13T09:19:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242392+0000", 2024-09-13T09:19:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0762e-05 2024-09-13T09:19:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242398+0000", 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1040000000000003e-06 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284506+0000", 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042107983000000002 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284564+0000", 2024-09-13T09:19:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.821e-05 2024-09-13T09:19:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284657+0000", 2024-09-13T09:19:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2588000000000002e-05 2024-09-13T09:19:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54991 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6840320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:19:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.446991668999999, 2024-09-13T09:19:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042489459, 2024-09-13T09:19:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54991 2024-09-13T09:19:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:19:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242406+0000", 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242412+0000", 2024-09-13T09:19:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5919999999999997e-06 2024-09-13T09:19:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242425+0000", 2024-09-13T09:19:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2713e-05 2024-09-13T09:19:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284694+0000", 2024-09-13T09:19:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042269558999999998 2024-09-13T09:19:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284784+0000", 2024-09-13T09:19:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9999000000000003e-05 2024-09-13T09:19:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284893+0000", 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108422 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54992 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6938624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.446965297999999, 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042636897999999999, 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54992 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242429+0000", 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242432+0000", 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7599999999999998e-06 2024-09-13T09:19:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242439+0000", 2024-09-13T09:19:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5760000000000002e-06 2024-09-13T09:19:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242448+0000", 2024-09-13T09:19:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-13T09:19:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284935+0000", 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042486949000000003 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284991+0000", 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6471000000000002e-05 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285067+0000", 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5543000000000005e-05 2024-09-13T09:19:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54993 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7036928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.446942420999999, 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042867959999999997, 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54993 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242452+0000", 2024-09-13T09:19:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242455+0000", 2024-09-13T09:19:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4360000000000001e-06 2024-09-13T09:19:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242463+0000", 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8220000000000007e-06 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242468+0000", 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9479999999999999e-06 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285107+0000", 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042639415999999999 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285196+0000", 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8721000000000007e-05 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285321+0000", 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012495 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54994 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7135232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:19:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.446915257000001, 2024-09-13T09:19:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043009622999999997, 2024-09-13T09:19:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54994 2024-09-13T09:19:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242482+0000", 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242486+0000", 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1049999999999997e-06 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242489+0000", 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.004e-06 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285354+0000", 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.04286479 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285413+0000", 2024-09-13T09:19:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9778000000000003e-05 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285489+0000", 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6141000000000002e-05 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54995 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7233536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:19:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.446903886000001, 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043166641999999998, 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54995 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:19:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242492+0000", 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.437e-06 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242496+0000", 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242498+0000", 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5950000000000001e-06 2024-09-13T09:19:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285528+0000", 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043029755000000003 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285585+0000", 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285658+0000", 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2713999999999995e-05 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54996 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7331840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.446894571000001, 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043333782000000001, 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54996 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242500+0000", 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242502+0000", 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3599999999999999e-06 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242505+0000", 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1439999999999999e-06 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242507+0000", 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.605e-06 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285698+0000", 2024-09-13T09:19:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043190911999999998 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285754+0000", 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5541999999999998e-05 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285834+0000", 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0426000000000006e-05 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54997 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.446885481999999, 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043496174999999998, 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54997 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242509+0000", 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242511+0000", 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.313e-06 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242514+0000", 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242516+0000", 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3999999999999999e-06 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285875+0000", 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043358578000000002 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285929+0000", 2024-09-13T09:19:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4602999999999999e-05 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286006+0000", 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6429999999999998e-05 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54998 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7528448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:19:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.446792869999999, 2024-09-13T09:19:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043666469999999999, 2024-09-13T09:19:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54998 2024-09-13T09:19:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242518+0000", 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999161 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242606+0000", 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7607000000000003e-05 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242617+0000", 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0882e-05 2024-09-13T09:19:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242624+0000", 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9009999999999997e-06 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286046+0000", 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043422308 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286106+0000", 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0285000000000003e-05 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286269+0000", 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000162578 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:55006 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8314880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.443404202, 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041461771000000001, 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 55006 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:19:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:19:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:19:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245993+0000", 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2869999999999999e-06 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245999+0000", 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0159999999999997e-06 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.246004+0000", 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287345+0000", 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041341217 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287398+0000", 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2840000000000002e-05 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287453+0000", 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4746000000000001e-05 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:68662 3.1d 3:be8be2fa:::rbd_data.118556027ff4.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.532348761000002, 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.044923689000000003, 2024-09-13T09:19:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 68662 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157045+0000", 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157052+0000", 2024-09-13T09:19:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8750000000000002e-06 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157066+0000", 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3815000000000001e-05 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157073+0000", 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.64e-06 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157145+0000", 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2517999999999999e-05 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157267+0000", 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012185599999999999 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.164012+0000", 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0067447660000000001 2024-09-13T09:19:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201805+0000", 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037793528999999999 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201881+0000", 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5678000000000003e-05 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201933+0000", 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2346999999999999e-05 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201970+0000", 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6532000000000002e-05 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:78869 3.5 3:a7c01924:::rbd_data.11853dd604c.000000000000000b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-13T09:19:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:59.731157+0000", 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.9582380490000002, 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062116486999999998, 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78869 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731157+0000", 2024-09-13T09:19:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731157+0000", 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731155+0000", 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731171+0000", 2024-09-13T09:19:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5543e-05 2024-09-13T09:19:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731202+0000", 2024-09-13T09:19:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1229999999999997e-05 2024-09-13T09:19:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731217+0000", 2024-09-13T09:19:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5173e-05 2024-09-13T09:19:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731336+0000", 2024-09-13T09:19:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119026 2024-09-13T09:19:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731566+0000", 2024-09-13T09:19:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022948399999999999 2024-09-13T09:19:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-13T09:19:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.742507+0000", 2024-09-13T09:19:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010941383000000001 2024-09-13T09:19:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.793038+0000", 2024-09-13T09:19:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050530670999999999 2024-09-13T09:19:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.793060+0000", 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2002999999999999e-05 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.793155+0000", 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4716000000000006e-05 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.793274+0000", 2024-09-13T09:19:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011882799999999999 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:79042 3.1c 3:3f2730ec:::rbd_data.11853dd604c.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:00.228566+0000", 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.460828915, 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066070161000000002, 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 79042 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228566+0000", 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228566+0000", 2024-09-13T09:19:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228564+0000", 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228575+0000", 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228605+0000", 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0426e-05 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228620+0000", 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4919999999999999e-05 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228720+0000", 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9889000000000005e-05 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228845+0000", 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012490600000000001 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.241066+0000", 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012220795 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.294366+0000", 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053300301000000001 2024-09-13T09:19:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.294481+0000", 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114629 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.294571+0000", 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0093000000000006e-05 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.294636+0000", 2024-09-13T09:19:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5605999999999995e-05 2024-09-13T09:19:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:02.807 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:02.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:19:02.809 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:02.802+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:02.910 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:02.906+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:02.983 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:02.983 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:02.983 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:02.983 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:02.997 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:19:03.046 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:03.042+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:03.112 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:03.109+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:03.173 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:03.174 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:03.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:19:03.213 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:03.209+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:03.314 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:03.309+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:03.317 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:03.317 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:03.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:03.317 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:03.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.318 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:50082 3.10 3:0ec43ecb:::rbd_object_map.1185172eb410:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T09:19:03.318 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:19:03.318 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.951962348000002, 2024-09-13T09:19:03.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80943783999999996, 2024-09-13T09:19:03.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:03.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 50082 2024-09-13T09:19:03.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362619+0000", 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362632+0000", 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3338e-05 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362657+0000", 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5207000000000001e-05 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362669+0000", 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1768e-05 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362738+0000", 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9486999999999994e-05 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362748+0000", 2024-09-13T09:19:03.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0010000000000001e-05 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169473+0000", 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80672482300000004 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169884+0000", 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041114299999999999 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:19:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.170206+0000", 2024-09-13T09:19:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032127700000000002 2024-09-13T09:19:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171827+0000", 2024-09-13T09:19:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016217880000000001 2024-09-13T09:19:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171871+0000", 2024-09-13T09:19:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171967+0000", 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5752000000000001e-05 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.172059+0000", 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2082000000000004e-05 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.325 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56772 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 4112384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.325 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:19:03.325 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.884274228999999, 2024-09-13T09:19:03.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054477114, 2024-09-13T09:19:03.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56772 2024-09-13T09:19:03.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430309+0000", 2024-09-13T09:19:03.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430312+0000", 2024-09-13T09:19:03.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.255e-06 2024-09-13T09:19:03.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430319+0000", 2024-09-13T09:19:03.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4760000000000001e-06 2024-09-13T09:19:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430324+0000", 2024-09-13T09:19:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9799999999999998e-06 2024-09-13T09:19:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484634+0000", 2024-09-13T09:19:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054310031000000002 2024-09-13T09:19:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484697+0000", 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3355999999999994e-05 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484787+0000", 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9350000000000003e-05 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.330 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56785 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5390336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.330 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:19:03.330 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880490747, 2024-09-13T09:19:03.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054655007999999998, 2024-09-13T09:19:03.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56785 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434094+0000", 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.471e-06 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434097+0000", 2024-09-13T09:19:03.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0740000000000001e-06 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434100+0000", 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7829999999999999e-06 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488422+0000", 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054321855000000002 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488563+0000", 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014154 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488748+0000", 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018461399999999999 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56786 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.334 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880481365000001, 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054950539999999999, 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56786 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:19:03.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434104+0000", 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4020000000000001e-06 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434107+0000", 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.072e-06 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434109+0000", 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.717e-06 2024-09-13T09:19:03.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488795+0000", 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054685661000000003 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488869+0000", 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4265999999999997e-05 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489053+0000", 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018374299999999999 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56787 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5586944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880471106000002, 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055115725999999997, 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.339 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56787 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:19:03.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434112+0000", 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434114+0000", 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.308e-06 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434117+0000", 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434119+0000", 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.588e-06 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489097+0000", 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054977824000000002 2024-09-13T09:19:03.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489146+0000", 2024-09-13T09:19:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8503e-05 2024-09-13T09:19:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489228+0000", 2024-09-13T09:19:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2694000000000005e-05 2024-09-13T09:19:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56788 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880462215000001, 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055437562000000003, 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56788 2024-09-13T09:19:03.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434121+0000", 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434123+0000", 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2190000000000001e-06 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434126+0000", 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:19:03.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434128+0000", 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8250000000000001e-06 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489293+0000", 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055164092999999997 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489428+0000", 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013572599999999999 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489559+0000", 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013086 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56789 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.348 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880453066000001, 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055674077000000002, 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56789 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:19:03.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:19:03.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:19:03.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434132+0000", 2024-09-13T09:19:03.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.302e-06 2024-09-13T09:19:03.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434136+0000", 2024-09-13T09:19:03.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434139+0000", 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489583+0000", 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055443994000000003 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489699+0000", 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116611 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489805+0000", 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105331 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.353 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56790 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.354 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:19:03.354 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880442690999999, 2024-09-13T09:19:03.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055716839999999997, 2024-09-13T09:19:03.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.354 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56790 2024-09-13T09:19:03.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.354 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:19:03.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434142+0000", 2024-09-13T09:19:03.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2559999999999999e-06 2024-09-13T09:19:03.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434145+0000", 2024-09-13T09:19:03.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.96e-06 2024-09-13T09:19:03.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434148+0000", 2024-09-13T09:19:03.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6479999999999999e-06 2024-09-13T09:19:03.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489851+0000", 2024-09-13T09:19:03.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055702792000000001 2024-09-13T09:19:03.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489858+0000", 2024-09-13T09:19:03.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3900000000000004e-06 2024-09-13T09:19:03.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.362 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.362 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56791 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880433919000001, 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055740029000000003, 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56791 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434151+0000", 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434154+0000", 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0139999999999999e-06 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434157+0000", 2024-09-13T09:19:03.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6699999999999998e-06 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489884+0000", 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055727511 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489890+0000", 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7200000000000003e-06 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56792 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880424990000002, 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055760363, 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56792 2024-09-13T09:19:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434163+0000", 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434167+0000", 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.292e-06 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434169+0000", 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489914+0000", 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055744415999999998 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489919+0000", 2024-09-13T09:19:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:19:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56793 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:19:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880411469999999, 2024-09-13T09:19:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055774049999999999, 2024-09-13T09:19:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56793 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434173+0000", 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-06 2024-09-13T09:19:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434177+0000", 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1070000000000001e-06 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434179+0000", 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.666e-06 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489941+0000", 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055762071000000003 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489946+0000", 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56794 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:19:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880402362000002, 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055791704999999997, 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56794 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:19:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434182+0000", 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1629999999999999e-06 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434185+0000", 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9950000000000001e-06 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434188+0000", 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489968+0000", 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055779789000000003 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489973+0000", 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2800000000000003e-06 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56795 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880393426000001, 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055809612000000001, 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56795 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434191+0000", 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.212e-06 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434194+0000", 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.078e-06 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434198+0000", 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9820000000000002e-06 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489995+0000", 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055796363000000002 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490000+0000", 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1839999999999998e-06 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56796 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880383165000001, 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055825859999999998, 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56796 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434200+0000", 2024-09-13T09:19:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434202+0000", 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.204e-06 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434205+0000", 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9689999999999999e-06 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434207+0000", 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490021+0000", 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055813929999999998 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490027+0000", 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.259e-06 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.387 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56797 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.880374446000001, 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055843649000000002, 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.388 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56797 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434210+0000", 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2389999999999999e-06 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434213+0000", 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:19:03.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434216+0000", 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7800000000000001e-06 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490048+0000", 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055831981000000003 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490053+0000", 2024-09-13T09:19:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8640000000000004e-06 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56807 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.878694287999998, 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054407720999999999, 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56807 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:19:03.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435889+0000", 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435891+0000", 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.217e-06 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435894+0000", 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.067e-06 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435896+0000", 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490292+0000", 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054395840000000001 2024-09-13T09:19:03.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490297+0000", 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56808 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.878685462, 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05442524, 2024-09-13T09:19:03.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56808 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435900+0000", 2024-09-13T09:19:03.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435903+0000", 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435905+0000", 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490319+0000", 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054413396000000003 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490324+0000", 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.400 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56809 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.878675405999999, 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054442786, 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56809 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435910+0000", 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435913+0000", 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1470000000000001e-06 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435915+0000", 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6769999999999999e-06 2024-09-13T09:19:03.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490346+0000", 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054430574000000002 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490351+0000", 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2700000000000004e-06 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56810 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.878666107000001, 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054459855000000001, 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56810 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.405 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435917+0000", 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435919+0000", 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.457e-06 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435924+0000", 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435929+0000", 2024-09-13T09:19:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8300000000000003e-06 2024-09-13T09:19:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490373+0000", 2024-09-13T09:19:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054443841999999999 2024-09-13T09:19:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490378+0000", 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8509999999999998e-06 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56811 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.878637006000002, 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054456683999999998, 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56811 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:19:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435946+0000", 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435949+0000", 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435956+0000", 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8719999999999996e-06 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435962+0000", 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000004e-06 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490398+0000", 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054436824000000002 2024-09-13T09:19:03.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490403+0000", 2024-09-13T09:19:03.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0710000000000002e-06 2024-09-13T09:19:03.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.413 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.413 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.413 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:03.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:19:03.415 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:03.413+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:03.516 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:03.513+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:03.544 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:03.545 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:03.545 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:03.545 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:03.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:19:03.617 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:03.613+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:03.625 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:19:03.625 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:19:03.625 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [2, 3, 4, 5, 7, 0, 1, 6] 2024-09-13T09:19:03.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-09-13T09:19:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:19:03.690 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:03.691 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:03.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:19:03.723 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:03.721+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:03.819 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:03.813+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:03.838 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:19:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:19:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 112.443598747, 2024-09-13T09:19:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077478195999999999, 2024-09-13T09:19:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:19:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:19:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:19:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:19:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:19:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392636+0000", 2024-09-13T09:19:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392651+0000", 2024-09-13T09:19:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:19:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392674+0000", 2024-09-13T09:19:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.264e-05 2024-09-13T09:19:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392691+0000", 2024-09-13T09:19:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7203000000000001e-05 2024-09-13T09:19:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392780+0000", 2024-09-13T09:19:03.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8671999999999994e-05 2024-09-13T09:19:03.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392794+0000", 2024-09-13T09:19:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-13T09:19:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.466664+0000", 2024-09-13T09:19:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073869913999999995 2024-09-13T09:19:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467139+0000", 2024-09-13T09:19:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047488599999999999 2024-09-13T09:19:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:19:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467664+0000", 2024-09-13T09:19:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052530600000000002 2024-09-13T09:19:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469125+0000", 2024-09-13T09:19:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001460432 2024-09-13T09:19:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469847+0000", 2024-09-13T09:19:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072197799999999996 2024-09-13T09:19:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470006+0000", 2024-09-13T09:19:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015955900000000001 2024-09-13T09:19:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470116+0000", 2024-09-13T09:19:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109836 2024-09-13T09:19:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:49389 3.d 3:b0b9272f:::rbd_data.1185e728d948.0000000000000025:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-13T09:19:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:19:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.753279970999998, 2024-09-13T09:19:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077194925999999997, 2024-09-13T09:19:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 49389 2024-09-13T09:19:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:19:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:19:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082955+0000", 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082967+0000", 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2609000000000001e-05 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082998+0000", 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0596000000000003e-05 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083013+0000", 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4766999999999999e-05 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083136+0000", 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012295599999999999 2024-09-13T09:19:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083360+0000", 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000224661 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.097720+0000", 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014359627 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.159975+0000", 2024-09-13T09:19:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062254728000000002 2024-09-13T09:19:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.159999+0000", 2024-09-13T09:19:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.427e-05 2024-09-13T09:19:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.160089+0000", 2024-09-13T09:19:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9895999999999994e-05 2024-09-13T09:19:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.160152+0000", 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2939000000000002e-05 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52426 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6209536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.030603691, 2024-09-13T09:19:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064408950000000006, 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52426 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805632+0000", 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805645+0000", 2024-09-13T09:19:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2469999999999999e-05 2024-09-13T09:19:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805654+0000", 2024-09-13T09:19:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7760000000000003e-06 2024-09-13T09:19:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805661+0000", 2024-09-13T09:19:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.114e-06 2024-09-13T09:19:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869801+0000", 2024-09-13T09:19:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064139777999999995 2024-09-13T09:19:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869918+0000", 2024-09-13T09:19:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117199 2024-09-13T09:19:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870042+0000", 2024-09-13T09:19:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124196 2024-09-13T09:19:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52427 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6307840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:19:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.030569760999999, 2024-09-13T09:19:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064720860000000005, 2024-09-13T09:19:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52427 2024-09-13T09:19:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:19:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805666+0000", 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805670+0000", 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2200000000000001e-06 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805678+0000", 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.32e-06 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805685+0000", 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.883e-06 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870103+0000", 2024-09-13T09:19:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064417962999999995 2024-09-13T09:19:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870241+0000", 2024-09-13T09:19:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138362 2024-09-13T09:19:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870388+0000", 2024-09-13T09:19:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146732 2024-09-13T09:19:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52428 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:19:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.030544031000002, 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064990774000000001, 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52428 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:19:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805692+0000", 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805695+0000", 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805704+0000", 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5299999999999996e-06 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805712+0000", 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870433+0000", 2024-09-13T09:19:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064720343999999999 2024-09-13T09:19:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870544+0000", 2024-09-13T09:19:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011108900000000001 2024-09-13T09:19:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870683+0000", 2024-09-13T09:19:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000139705 2024-09-13T09:19:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52429 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6504448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:19:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.030503735, 2024-09-13T09:19:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065238731999999994, 2024-09-13T09:19:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52429 2024-09-13T09:19:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:19:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:19:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805732+0000", 2024-09-13T09:19:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805736+0000", 2024-09-13T09:19:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8600000000000003e-06 2024-09-13T09:19:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805746+0000", 2024-09-13T09:19:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6860000000000001e-06 2024-09-13T09:19:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805753+0000", 2024-09-13T09:19:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:19:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870724+0000", 2024-09-13T09:19:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064970450999999999 2024-09-13T09:19:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870836+0000", 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112796 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870972+0000", 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135382 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52430 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6602752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:19:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.030477509000001, 2024-09-13T09:19:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065499180000000004, 2024-09-13T09:19:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52430 2024-09-13T09:19:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:19:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:19:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:19:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805762+0000", 2024-09-13T09:19:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:19:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805773+0000", 2024-09-13T09:19:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1496e-05 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805780+0000", 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871018+0000", 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065237396000000003 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871124+0000", 2024-09-13T09:19:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106469 2024-09-13T09:19:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871258+0000", 2024-09-13T09:19:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134175 2024-09-13T09:19:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52431 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6701056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.030450654999999, 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065755623999999999, 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52431 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805785+0000", 2024-09-13T09:19:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805789+0000", 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2160000000000002e-06 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805797+0000", 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4870000000000002e-06 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805804+0000", 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871298+0000", 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065494259999999999 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871405+0000", 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106512 2024-09-13T09:19:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871542+0000", 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013683599999999999 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52432 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6799360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.030385956, 2024-09-13T09:19:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065976444999999995, 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52432 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:19:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805850+0000", 2024-09-13T09:19:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805855+0000", 2024-09-13T09:19:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8670000000000002e-06 2024-09-13T09:19:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805867+0000", 2024-09-13T09:19:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2597e-05 2024-09-13T09:19:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805876+0000", 2024-09-13T09:19:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.833e-06 2024-09-13T09:19:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871584+0000", 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065707765000000001 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871690+0000", 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105832 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871827+0000", 2024-09-13T09:19:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013744500000000001 2024-09-13T09:19:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52440 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7585792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:19:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.026262143, 2024-09-13T09:19:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064276712999999999, 2024-09-13T09:19:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52440 2024-09-13T09:19:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809974+0000", 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809976+0000", 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8920000000000001e-06 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809981+0000", 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5170000000000001e-06 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809985+0000", 2024-09-13T09:19:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9539999999999998e-06 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.873936+0000", 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063951470999999996 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874076+0000", 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014022199999999999 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874251+0000", 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017497500000000001 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52441 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7684096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.026248764000002, 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064567507999999996, 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52441 2024-09-13T09:19:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:19:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809989+0000", 2024-09-13T09:19:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8589999999999999e-06 2024-09-13T09:19:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809994+0000", 2024-09-13T09:19:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:19:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809999+0000", 2024-09-13T09:19:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:19:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874317+0000", 2024-09-13T09:19:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064318742999999998 2024-09-13T09:19:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874423+0000", 2024-09-13T09:19:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106116 2024-09-13T09:19:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874555+0000", 2024-09-13T09:19:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013196700000000001 2024-09-13T09:19:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52442 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7782400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:19:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.026234761000001, 2024-09-13T09:19:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064804082999999998, 2024-09-13T09:19:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52442 2024-09-13T09:19:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:19:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:19:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:19:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810003+0000", 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8360000000000001e-06 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810009+0000", 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0220000000000001e-06 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810013+0000", 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9879999999999998e-06 2024-09-13T09:19:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874593+0000", 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064579851999999993 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874688+0000", 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4581999999999997e-05 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874806+0000", 2024-09-13T09:19:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118108 2024-09-13T09:19:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52443 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7880704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:19:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.026219865000002, 2024-09-13T09:19:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065175650000000002, 2024-09-13T09:19:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52443 2024-09-13T09:19:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:19:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810018+0000", 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.798e-06 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810023+0000", 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7829999999999998e-06 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810027+0000", 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8890000000000004e-06 2024-09-13T09:19:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874886+0000", 2024-09-13T09:19:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064859362000000004 2024-09-13T09:19:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875029+0000", 2024-09-13T09:19:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014311900000000001 2024-09-13T09:19:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875192+0000", 2024-09-13T09:19:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000163012 2024-09-13T09:19:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52444 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7979008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:19:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.026206286000001, 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065471830999999994, 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52444 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:19:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:19:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:19:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810032+0000", 2024-09-13T09:19:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8199999999999999e-06 2024-09-13T09:19:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810036+0000", 2024-09-13T09:19:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5510000000000001e-06 2024-09-13T09:19:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810041+0000", 2024-09-13T09:19:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0679999999999998e-06 2024-09-13T09:19:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875256+0000", 2024-09-13T09:19:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065215765999999994 2024-09-13T09:19:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875366+0000", 2024-09-13T09:19:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109639 2024-09-13T09:19:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875502+0000", 2024-09-13T09:19:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013630399999999999 2024-09-13T09:19:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52445 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.02619297, 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065733503999999998, 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52445 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810043+0000", 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810045+0000", 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.813e-06 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810050+0000", 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5940000000000004e-06 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810054+0000", 2024-09-13T09:19:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439999999999998e-06 2024-09-13T09:19:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875543+0000", 2024-09-13T09:19:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065489393000000007 2024-09-13T09:19:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875658+0000", 2024-09-13T09:19:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115302 2024-09-13T09:19:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875777+0000", 2024-09-13T09:19:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118776 2024-09-13T09:19:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52446 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8175616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.026179640999999, 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065969204000000004, 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52446 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:19:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:19:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:19:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810059+0000", 2024-09-13T09:19:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.84e-06 2024-09-13T09:19:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810063+0000", 2024-09-13T09:19:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4329999999999997e-06 2024-09-13T09:19:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810067+0000", 2024-09-13T09:19:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8800000000000001e-06 2024-09-13T09:19:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875813+0000", 2024-09-13T09:19:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065745910000000005 2024-09-13T09:19:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875903+0000", 2024-09-13T09:19:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0665e-05 2024-09-13T09:19:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876026+0000", 2024-09-13T09:19:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122836 2024-09-13T09:19:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52447 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8273920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:19:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.026145400000001, 2024-09-13T09:19:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066176867, 2024-09-13T09:19:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52447 2024-09-13T09:19:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:19:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:19:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:19:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810093+0000", 2024-09-13T09:19:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6089999999999999e-06 2024-09-13T09:19:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810099+0000", 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1539999999999999e-06 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810104+0000", 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4630000000000003e-06 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876062+0000", 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065957672999999994 2024-09-13T09:19:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876153+0000", 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1119000000000006e-05 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876268+0000", 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115431 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52448 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:19:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.026129231999999, 2024-09-13T09:19:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066534826000000005, 2024-09-13T09:19:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52448 2024-09-13T09:19:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:19:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:19:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810109+0000", 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0169999999999999e-06 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810115+0000", 2024-09-13T09:19:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:19:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810120+0000", 2024-09-13T09:19:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9620000000000001e-06 2024-09-13T09:19:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876349+0000", 2024-09-13T09:19:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066229213999999995 2024-09-13T09:19:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876484+0000", 2024-09-13T09:19:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013503199999999999 2024-09-13T09:19:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876642+0000", 2024-09-13T09:19:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158228 2024-09-13T09:19:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52452 3.d 3:b5f40919:::rbd_data.118567e41306.0000000000000055:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:19:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.026113393999999, 2024-09-13T09:19:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070307575999999997, 2024-09-13T09:19:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52452 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810126+0000", 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:19:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810132+0000", 2024-09-13T09:19:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7349999999999999e-06 2024-09-13T09:19:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810136+0000", 2024-09-13T09:19:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2069999999999997e-06 2024-09-13T09:19:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876306+0000", 2024-09-13T09:19:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066169943999999994 2024-09-13T09:19:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876520+0000", 2024-09-13T09:19:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021377500000000001 2024-09-13T09:19:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.877399+0000", 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087836900000000005 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880158+0000", 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002759657 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880212+0000", 2024-09-13T09:19:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3551999999999998e-05 2024-09-13T09:19:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880315+0000", 2024-09-13T09:19:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010306099999999999 2024-09-13T09:19:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880431+0000", 2024-09-13T09:19:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011603300000000001 2024-09-13T09:19:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:78930 3.13 3:cc588144:::rbd_data.11853dd604c.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-13T09:19:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:59.956179+0000", 2024-09-13T09:19:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.8800577560000002, 2024-09-13T09:19:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067694713000000004, 2024-09-13T09:19:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78930 2024-09-13T09:19:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956179+0000", 2024-09-13T09:19:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956179+0000", 2024-09-13T09:19:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956177+0000", 2024-09-13T09:19:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:19:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956192+0000", 2024-09-13T09:19:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5173e-05 2024-09-13T09:19:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956223+0000", 2024-09-13T09:19:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1056000000000002e-05 2024-09-13T09:19:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956242+0000", 2024-09-13T09:19:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8916999999999999e-05 2024-09-13T09:19:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956341+0000", 2024-09-13T09:19:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9011999999999993e-05 2024-09-13T09:19:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956470+0000", 2024-09-13T09:19:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129603 2024-09-13T09:19:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.969554+0000", 2024-09-13T09:19:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013083539999999999 2024-09-13T09:19:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.023628+0000", 2024-09-13T09:19:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054073884000000003 2024-09-13T09:19:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.023715+0000", 2024-09-13T09:19:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7056000000000001e-05 2024-09-13T09:19:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.023809+0000", 2024-09-13T09:19:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4382000000000005e-05 2024-09-13T09:19:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.023874+0000", 2024-09-13T09:19:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4403e-05 2024-09-13T09:19:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:03.971 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:03.972 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-09-13T09:19:03.949+0000 7f1d86e9f640 -1 mon.a@0(leader).osd e100 definitely_dead 0 2024-09-13T09:19:03.973 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:03.965+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:03.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:19:04.073 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:04.069+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:04.102 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:04.102 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:04.102 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:04.103 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:04.118 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:19:04.203 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:04.201+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:04.278 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:04.273+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:04.285 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:04.285 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:04.285 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:04.285 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:04.285 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:04.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:19:04.382 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:04.377+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:04.470 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:04.470 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:04.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3:ed699f0f:::rbd_data.118591b29fad.000000000000008c:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 81.607196447000007, 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94878089700000001, 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:19:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.472 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:04.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:04.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:19:04.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810654+0000", 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94834837599999999 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810755+0000", 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100987 2024-09-13T09:19:04.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810828+0000", 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2751999999999997e-05 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19465 3.10 3:0e3a5b30:::rbd_data.1185b72576e4.0000000000000062:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 67.521351710000005, 2024-09-13T09:19:04.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97489159000000003, 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19465 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947890+0000", 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947899+0000", 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6840000000000008e-06 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947917+0000", 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7731e-05 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947925+0000", 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4189999999999996e-06 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947990+0000", 2024-09-13T09:19:04.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5909999999999997e-05 2024-09-13T09:19:04.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:04.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947998+0000", 2024-09-13T09:19:04.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4449999999999999e-06 2024-09-13T09:19:04.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.919994+0000", 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97199595900000002 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920258+0000", 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026439500000000001 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920629+0000", 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037096299999999998 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.921797+0000", 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001167721 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922544+0000", 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00074661200000000001 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922628+0000", 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4016000000000003e-05 2024-09-13T09:19:04.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922783+0000", 2024-09-13T09:19:04.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015585499999999999 2024-09-13T09:19:04.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.483 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.483 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51771 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.483 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:04.483 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724615236999998, 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083783107999999995, 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51771 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:04.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744631+0000", 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.943e-06 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744639+0000", 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744646+0000", 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5860000000000001e-06 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828151+0000", 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083505732999999999 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828286+0000", 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013457200000000001 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828411+0000", 2024-09-13T09:19:04.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012564999999999999 2024-09-13T09:19:04.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.488 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.488 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51772 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.488 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:04.488 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724592140999999, 2024-09-13T09:19:04.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083939185999999999, 2024-09-13T09:19:04.488 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.488 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51772 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:04.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:04.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744654+0000", 2024-09-13T09:19:04.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2440000000000002e-06 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744662+0000", 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3659999999999992e-06 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744669+0000", 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6529999999999997e-06 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828455+0000", 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083785707000000001 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828512+0000", 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7549000000000001e-05 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828591+0000", 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8274000000000003e-05 2024-09-13T09:19:04.492 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.493 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.493 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.493 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51773 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.493 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:04.493 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724568484999999, 2024-09-13T09:19:04.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084139096999999996, 2024-09-13T09:19:04.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.493 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.493 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.494 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51773 2024-09-13T09:19:04.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.495 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:04.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:04.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744674+0000", 2024-09-13T09:19:04.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744677+0000", 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9399999999999998e-06 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744689+0000", 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.153e-05 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744705+0000", 2024-09-13T09:19:04.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5679e-05 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828658+0000", 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083953190999999996 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828734+0000", 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5784999999999994e-05 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828814+0000", 2024-09-13T09:19:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579000000000002e-05 2024-09-13T09:19:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51774 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724531122999998, 2024-09-13T09:19:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084301713, 2024-09-13T09:19:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51774 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744723+0000", 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0832000000000001e-05 2024-09-13T09:19:04.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744733+0000", 2024-09-13T09:19:04.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031e-05 2024-09-13T09:19:04.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744740+0000", 2024-09-13T09:19:04.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0500000000000003e-06 2024-09-13T09:19:04.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828867+0000", 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084127417999999995 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828932+0000", 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4202000000000006e-05 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829014+0000", 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-13T09:19:04.503 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51775 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724497451000001, 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084465871999999997, 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.504 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51775 2024-09-13T09:19:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:04.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744749+0000", 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5200000000000002e-06 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744757+0000", 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1100000000000003e-06 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744764+0000", 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8430000000000004e-06 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829058+0000", 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084293899000000005 2024-09-13T09:19:04.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829111+0000", 2024-09-13T09:19:04.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3315e-05 2024-09-13T09:19:04.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829212+0000", 2024-09-13T09:19:04.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010079200000000001 2024-09-13T09:19:04.508 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.508 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.508 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51776 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724473739, 2024-09-13T09:19:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084631102999999999, 2024-09-13T09:19:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51776 2024-09-13T09:19:04.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.510 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:04.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:04.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744769+0000", 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744772+0000", 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.14e-06 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744781+0000", 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.276e-06 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744788+0000", 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0269999999999998e-06 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829254+0000", 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084466325999999994 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829310+0000", 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5852e-05 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829401+0000", 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1075000000000005e-05 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51777 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724418471, 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084901272, 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51777 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744824+0000", 2024-09-13T09:19:04.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:04.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744828+0000", 2024-09-13T09:19:04.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:19:04.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744840+0000", 2024-09-13T09:19:04.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1997000000000001e-05 2024-09-13T09:19:04.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744847+0000", 2024-09-13T09:19:04.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0659999999999998e-06 2024-09-13T09:19:04.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829460+0000", 2024-09-13T09:19:04.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084613387999999998 2024-09-13T09:19:04.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829555+0000", 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4999000000000003e-05 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829726+0000", 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017100899999999999 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51778 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.518 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724366026999999, 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085026334999999995, 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51778 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:04.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:04.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744877+0000", 2024-09-13T09:19:04.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744881+0000", 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5999999999999998e-06 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744892+0000", 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1732999999999999e-05 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744899+0000", 2024-09-13T09:19:04.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0759999999999997e-06 2024-09-13T09:19:04.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829772+0000", 2024-09-13T09:19:04.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084872633000000003 2024-09-13T09:19:04.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829826+0000", 2024-09-13T09:19:04.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4330000000000003e-05 2024-09-13T09:19:04.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829904+0000", 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7558999999999994e-05 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51779 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724337458000001, 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085200608999999997, 2024-09-13T09:19:04.523 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51779 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:04.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744905+0000", 2024-09-13T09:19:04.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744909+0000", 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3500000000000001e-06 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744917+0000", 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3129999999999999e-06 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744924+0000", 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6059999999999999e-06 2024-09-13T09:19:04.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829976+0000", 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085051775999999996 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830030+0000", 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4404000000000003e-05 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830107+0000", 2024-09-13T09:19:04.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6796000000000001e-05 2024-09-13T09:19:04.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.528 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.528 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51780 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.528 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:04.528 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724313864999999, 2024-09-13T09:19:04.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085367775000000007, 2024-09-13T09:19:04.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.528 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51780 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744929+0000", 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744932+0000", 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744940+0000", 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1130000000000001e-06 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744947+0000", 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830148+0000", 2024-09-13T09:19:04.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085201099000000002 2024-09-13T09:19:04.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830205+0000", 2024-09-13T09:19:04.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6623999999999999e-05 2024-09-13T09:19:04.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830297+0000", 2024-09-13T09:19:04.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2693000000000002e-05 2024-09-13T09:19:04.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.533 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.533 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51781 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.533 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:04.533 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724290829000001, 2024-09-13T09:19:04.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085544856000000002, 2024-09-13T09:19:04.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.533 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51781 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744952+0000", 2024-09-13T09:19:04.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744955+0000", 2024-09-13T09:19:04.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0960000000000001e-06 2024-09-13T09:19:04.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745048+0000", 2024-09-13T09:19:04.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2806999999999994e-05 2024-09-13T09:19:04.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745058+0000", 2024-09-13T09:19:04.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0132999999999999e-05 2024-09-13T09:19:04.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830338+0000", 2024-09-13T09:19:04.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085279761999999995 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830420+0000", 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1551000000000006e-05 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830498+0000", 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8103999999999996e-05 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51782 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1794048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724178906999999, 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085605889000000004, 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51782 2024-09-13T09:19:04.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745064+0000", 2024-09-13T09:19:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745068+0000", 2024-09-13T09:19:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.523e-06 2024-09-13T09:19:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745076+0000", 2024-09-13T09:19:04.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.354e-06 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745091+0000", 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5274999999999999e-05 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830538+0000", 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085447238999999994 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830591+0000", 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2481000000000001e-05 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830671+0000", 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9716999999999996e-05 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51783 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1892352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.542 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724146261000001, 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085732662000000001, 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51783 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745100+0000", 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745108+0000", 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4360000000000002e-06 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745115+0000", 2024-09-13T09:19:04.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9269999999999999e-06 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830707+0000", 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085591133999999999 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830758+0000", 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1180000000000001e-05 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830830+0000", 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2268999999999995e-05 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51784 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.724114293, 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085902834999999997, 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.548 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51784 2024-09-13T09:19:04.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.548 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:04.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745132+0000", 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.36e-06 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745143+0000", 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745150+0000", 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0400000000000004e-06 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830870+0000", 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085720086000000001 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830955+0000", 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5321000000000005e-05 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831032+0000", 2024-09-13T09:19:04.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6811999999999996e-05 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51785 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.723559397999999, 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085542517999999998, 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.553 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51785 2024-09-13T09:19:04.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.553 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:04.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:04.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745160+0000", 2024-09-13T09:19:04.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994755 2024-09-13T09:19:04.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745697+0000", 2024-09-13T09:19:04.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00053713000000000003 2024-09-13T09:19:04.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745731+0000", 2024-09-13T09:19:04.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3767999999999999e-05 2024-09-13T09:19:04.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745754+0000", 2024-09-13T09:19:04.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3748999999999999e-05 2024-09-13T09:19:04.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831075+0000", 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085321067 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831130+0000", 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4156999999999997e-05 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831227+0000", 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7093e-05 2024-09-13T09:19:04.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.557 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.557 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51789 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.557 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:04.557 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.720866642000001, 2024-09-13T09:19:04.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083633549000000001, 2024-09-13T09:19:04.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.557 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.557 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51789 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748376+0000", 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748380+0000", 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7529999999999999e-06 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748392+0000", 2024-09-13T09:19:04.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2376e-05 2024-09-13T09:19:04.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748399+0000", 2024-09-13T09:19:04.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0199999999999997e-06 2024-09-13T09:19:04.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831866+0000", 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083466636999999996 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831933+0000", 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7144000000000006e-05 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832010+0000", 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7352000000000001e-05 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51790 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.720838076, 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083812485000000006, 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.562 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51790 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748405+0000", 2024-09-13T09:19:04.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748408+0000", 2024-09-13T09:19:04.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3859999999999999e-06 2024-09-13T09:19:04.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748417+0000", 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748424+0000", 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832055+0000", 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083631469 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832129+0000", 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4240999999999996e-05 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832218+0000", 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8562999999999999e-05 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51791 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2678784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:04.566 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.720813657000001, 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083959632000000006, 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51791 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:04.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748429+0000", 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748432+0000", 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0800000000000002e-06 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748441+0000", 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9120000000000004e-06 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748470+0000", 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569e-05 2024-09-13T09:19:04.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832260+0000", 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083790586 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832317+0000", 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6054999999999998e-05 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:04.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832390+0000", 2024-09-13T09:19:04.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3033000000000004e-05 2024-09-13T09:19:04.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.571 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:04.571 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:04.571 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:04.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:19:04.573 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:04.569+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:04.666 INFO:teuthology.orchestra.run.smithi032.stderr:marked out osd.6. 2024-09-13T09:19:04.673 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:04.669+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:04.680 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:19:04.765 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:04.765 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:04.765 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:04.765 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:04.774 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:04.769+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:04.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:19:04.875 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:04.869+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:04.899 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:04.899 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:04.899 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:04.899 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:04.899 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:04.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:19:04.976 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:04.969+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:05.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-13T09:19:05.005+0000 7f303e7fc640 0 --2- 172.21.15.32:0/3136686153 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30400ad500 0x7f30400adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T09:19:05.039 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:05.039 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:05.043 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:05.043 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:05.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:05.044 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:05.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.044 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:19:05.044 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:05.044 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.967248275999999, 2024-09-13T09:19:05.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:19:05.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.044 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:05.044 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.044 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.044 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.044 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:19:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:19:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:19:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:19:05.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:19:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:19:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:19:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:19:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:19:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.050 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56102 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5439488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.624407218999998, 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060495140000000003, 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56102 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416845+0000", 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416853+0000", 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416862+0000", 2024-09-13T09:19:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1959999999999997e-06 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416868+0000", 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9719999999999997e-06 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477088+0000", 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060219898000000001 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477154+0000", 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5605000000000006e-05 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477341+0000", 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018687599999999999 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56103 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5537792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.055 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.624379155, 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060576972999999999, 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56103 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416873+0000", 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416876+0000", 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.993e-06 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416883+0000", 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.477e-06 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416888+0000", 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2580000000000003e-06 2024-09-13T09:19:05.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477357+0000", 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060469055000000001 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477396+0000", 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9402e-05 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477451+0000", 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4262000000000003e-05 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56104 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5636096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.624360663000001, 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060778881999999999, 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56104 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:05.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416894+0000", 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4049999999999998e-06 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416900+0000", 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.959e-06 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416905+0000", 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477465+0000", 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060559460000000002 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477521+0000", 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5813000000000002e-05 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477671+0000", 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150366 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56105 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5734400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.624316451999999, 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060898832, 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56105 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:05.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416939+0000", 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3550000000000001e-06 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416947+0000", 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0949999999999996e-06 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416953+0000", 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6790000000000002e-06 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477708+0000", 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060755454 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477760+0000", 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1589999999999999e-05 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477835+0000", 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5394000000000004e-05 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56106 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5832704~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.624295135000001, 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061142999000000003, 2024-09-13T09:19:05.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56106 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416957+0000", 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416960+0000", 2024-09-13T09:19:05.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6979999999999998e-06 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416967+0000", 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6919999999999996e-06 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416973+0000", 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5679999999999996e-06 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477867+0000", 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060894082000000002 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477924+0000", 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7015999999999998e-05 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478101+0000", 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017648300000000001 2024-09-13T09:19:05.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.074 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.074 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56107 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5931008~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.074 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:05.074 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.623966521, 2024-09-13T09:19:05.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060956639999999999, 2024-09-13T09:19:05.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56107 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416977+0000", 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996905 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417291+0000", 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031392799999999999 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417311+0000", 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0343000000000002e-05 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417329+0000", 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7612e-05 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478123+0000", 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060794421000000001 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478174+0000", 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0614e-05 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478243+0000", 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9226000000000004e-05 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.078 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56108 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 6029312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.623907036999999, 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061163618000000003, 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56108 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417345+0000", 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417349+0000", 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7419999999999999e-06 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417367+0000", 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7714999999999999e-05 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417382+0000", 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5396999999999998e-05 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478270+0000", 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060888022999999999 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478322+0000", 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2234999999999997e-05 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478509+0000", 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187109 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56121 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7307264~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.619923543999999, 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060455939, 2024-09-13T09:19:05.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56121 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421332+0000", 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:19:05.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421358+0000", 2024-09-13T09:19:05.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6427999999999998e-05 2024-09-13T09:19:05.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421374+0000", 2024-09-13T09:19:05.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6137000000000002e-05 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481604+0000", 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060229528999999997 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481680+0000", 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6089999999999998e-05 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481785+0000", 2024-09-13T09:19:05.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105259 2024-09-13T09:19:05.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.088 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.088 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56122 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7405568~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.088 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:05.088 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.619850040999999, 2024-09-13T09:19:05.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060604366999999999, 2024-09-13T09:19:05.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56122 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421402+0000", 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421406+0000", 2024-09-13T09:19:05.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.337e-06 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421415+0000", 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421434+0000", 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8617000000000001e-05 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481827+0000", 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060393883000000002 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481910+0000", 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2657999999999994e-05 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482007+0000", 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7057000000000003e-05 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.092 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56123 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7503872~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.619813503, 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060838125999999999, 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.093 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56123 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421450+0000", 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1544e-05 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421463+0000", 2024-09-13T09:19:05.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421468+0000", 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6250000000000004e-06 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482066+0000", 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060597713999999997 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482163+0000", 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7112000000000001e-05 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482277+0000", 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114274 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.097 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56124 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7602176~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.619779548, 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061054929000000001, 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56124 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421476+0000", 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2059999999999999e-06 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421484+0000", 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7889999999999995e-06 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421489+0000", 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482321+0000", 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060831477000000002 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482403+0000", 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2776999999999997e-05 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482528+0000", 2024-09-13T09:19:05.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012491600000000001 2024-09-13T09:19:05.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.102 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.102 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56125 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7700480~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.102 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:05.102 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.619759251000001, 2024-09-13T09:19:05.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061273721000000003, 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56125 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421493+0000", 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421496+0000", 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.666e-06 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421503+0000", 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421508+0000", 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0919999999999997e-06 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482575+0000", 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061066339999999997 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482659+0000", 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4158000000000003e-05 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482767+0000", 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010849 2024-09-13T09:19:05.106 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.107 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.107 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.107 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56126 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7798784~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.107 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:05.107 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.619739885000001, 2024-09-13T09:19:05.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061489349999999998, 2024-09-13T09:19:05.107 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.107 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56126 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421531+0000", 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8218000000000001e-05 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421538+0000", 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.345e-06 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421544+0000", 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482813+0000", 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061268959999999997 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482900+0000", 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6858000000000001e-05 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483002+0000", 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102599 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.111 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56127 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7897088~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.112 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:05.112 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.619704144, 2024-09-13T09:19:05.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061729777, 2024-09-13T09:19:05.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.112 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.112 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.112 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.112 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56127 2024-09-13T09:19:05.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.112 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421548+0000", 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421551+0000", 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421559+0000", 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4499999999999998e-06 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421564+0000", 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483067+0000", 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061503073999999998 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483153+0000", 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483278+0000", 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012522499999999999 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56319 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1613824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.613946398, 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060599663999999998, 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.116 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56319 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427309+0000", 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.672e-06 2024-09-13T09:19:05.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427316+0000", 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6850000000000001e-06 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427322+0000", 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487756+0000", 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060434817000000002 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487825+0000", 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8553000000000006e-05 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487906+0000", 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1067000000000002e-05 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56320 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1712128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.613926584000001, 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060803930999999999, 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.121 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56320 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427328+0000", 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427379+0000", 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0343999999999998e-05 2024-09-13T09:19:05.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427385+0000", 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3969999999999999e-06 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487941+0000", 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060555805999999997 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488034+0000", 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488130+0000", 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6348999999999995e-05 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56321 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1810432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:05.125 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.613861647, 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060999937999999997, 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56321 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:05.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427394+0000", 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.721e-06 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427403+0000", 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7110000000000001e-06 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427409+0000", 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9710000000000001e-06 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488183+0000", 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060774003 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488291+0000", 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010786899999999999 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488391+0000", 2024-09-13T09:19:05.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100142 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56322 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1908736~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.613515532000001, 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060864109999999999, 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.130 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56322 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427433+0000", 2024-09-13T09:19:05.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996958 2024-09-13T09:19:05.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427742+0000", 2024-09-13T09:19:05.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000309084 2024-09-13T09:19:05.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427754+0000", 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1354e-05 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427761+0000", 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8059999999999995e-06 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488429+0000", 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060667139000000002 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488496+0000", 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7595000000000006e-05 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488601+0000", 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105257 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56323 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 2007040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.612909837, 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060546618000000003, 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56323 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:05.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427767+0000", 2024-09-13T09:19:05.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994245 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428348+0000", 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00058032600000000004 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428360+0000", 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2103000000000001e-05 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428367+0000", 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1949999999999997e-06 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488650+0000", 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060282626999999998 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488745+0000", 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5365999999999995e-05 2024-09-13T09:19:05.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488890+0000", 2024-09-13T09:19:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000144506 2024-09-13T09:19:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:05.139 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:05.139 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:05.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:19:05.141 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:05.141 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:05.142 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:05.142 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:05.142 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:05.143 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:05.144 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:05.144 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:05.144 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:05.137+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:05.158 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:05.158 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:05.182 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:05.182 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:05.247 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:05.245+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:05.277 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:05.277 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:05.291 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:19:05.379 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:05.377+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:05.443 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:05.441+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:05.461 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:05.461 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:05.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:19:05.567 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:05.565+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:05.640 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:05.640 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:05.645 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:05.641+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:05.656 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:19:05.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:19:05.745 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:05.741+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:05.821 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:05.821 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:05.837 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:19:05.847 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:05.841+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:05.948 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:05.945+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:05.994 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:05.994 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:06.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:19:06.051 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:06.049+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:06.150 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:06.145+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:06.167 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:06.167 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:06.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:19:06.251 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:06.245+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:06.315 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:06.315 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:06.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:19:06.374 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:06.369+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:06.453 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:06.449+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:06.500 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:06.500 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:06.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:19:06.562 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:06.557+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:06.634 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:06.635 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:06.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:19:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-13T09:19:06.693 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:06.689+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:06.755 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:06.749+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:06.827 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:06.827 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:06.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:19:06.890 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:06.885+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:06.957 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:06.953+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:07.027 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:07.027 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:07.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:19:07.089 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:07.085+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:07.159 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:07.157+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:07.169 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:07.169 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:07.185 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:19:07.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:19:07.260 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:07.257+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:07.356 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:07.356 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:07.358 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:07.359 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:07.360 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:07.357+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:07.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:19:07.376 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:07.376 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:07.393 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:07.409 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:07.427 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:07.444 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:07.461 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:07.457+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:07.463 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:07.484 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:07.496 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:07.496 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:07.511 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:19:07.562 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:07.557+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:07.631 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:07.631 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:07.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:07.631 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:07.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.632 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:50082 3.10 3:0ec43ecb:::rbd_object_map.1185172eb410:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T09:19:07.632 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:19:07.632 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.266941596000002, 2024-09-13T09:19:07.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80943783999999996, 2024-09-13T09:19:07.632 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.632 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:07.632 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.632 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.632 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 50082 2024-09-13T09:19:07.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.633 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:19:07.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362621+0000", 2024-09-13T09:19:07.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362619+0000", 2024-09-13T09:19:07.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:07.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362632+0000", 2024-09-13T09:19:07.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3338e-05 2024-09-13T09:19:07.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362657+0000", 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5207000000000001e-05 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362669+0000", 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1768e-05 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362738+0000", 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9486999999999994e-05 2024-09-13T09:19:07.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:07.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:30.362748+0000", 2024-09-13T09:19:07.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0010000000000001e-05 2024-09-13T09:19:07.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169473+0000", 2024-09-13T09:19:07.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.80672482300000004 2024-09-13T09:19:07.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:07.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.169884+0000", 2024-09-13T09:19:07.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041114299999999999 2024-09-13T09:19:07.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:19:07.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.170206+0000", 2024-09-13T09:19:07.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032127700000000002 2024-09-13T09:19:07.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:07.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171827+0000", 2024-09-13T09:19:07.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016217880000000001 2024-09-13T09:19:07.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:07.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171871+0000", 2024-09-13T09:19:07.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:19:07.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:07.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.171967+0000", 2024-09-13T09:19:07.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5752000000000001e-05 2024-09-13T09:19:07.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:31.172059+0000", 2024-09-13T09:19:07.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2082000000000004e-05 2024-09-13T09:19:07.639 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.639 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.640 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56772 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 4112384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.640 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:19:07.641 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.199253476999999, 2024-09-13T09:19:07.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054477114, 2024-09-13T09:19:07.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:07.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56772 2024-09-13T09:19:07.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.641 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430310+0000", 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430309+0000", 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:07.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430312+0000", 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.255e-06 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430319+0000", 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4760000000000001e-06 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.430324+0000", 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9799999999999998e-06 2024-09-13T09:19:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484634+0000", 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054310031000000002 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484697+0000", 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3355999999999994e-05 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.484787+0000", 2024-09-13T09:19:07.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9350000000000003e-05 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56785 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5390336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195469995000003, 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054655007999999998, 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56785 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434093+0000", 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434094+0000", 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.471e-06 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434097+0000", 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0740000000000001e-06 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434100+0000", 2024-09-13T09:19:07.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7829999999999999e-06 2024-09-13T09:19:07.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488422+0000", 2024-09-13T09:19:07.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054321855000000002 2024-09-13T09:19:07.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:07.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488563+0000", 2024-09-13T09:19:07.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014154 2024-09-13T09:19:07.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488748+0000", 2024-09-13T09:19:07.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018461399999999999 2024-09-13T09:19:07.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.649 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56786 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195460613000002, 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054950539999999999, 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56786 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434102+0000", 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434104+0000", 2024-09-13T09:19:07.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4020000000000001e-06 2024-09-13T09:19:07.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434107+0000", 2024-09-13T09:19:07.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.072e-06 2024-09-13T09:19:07.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434109+0000", 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.717e-06 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488795+0000", 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054685661000000003 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.488869+0000", 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4265999999999997e-05 2024-09-13T09:19:07.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489053+0000", 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018374299999999999 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56787 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5586944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195450354000002, 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055115725999999997, 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:07.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56787 2024-09-13T09:19:07.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.655 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:19:07.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434113+0000", 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434112+0000", 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434114+0000", 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.308e-06 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434117+0000", 2024-09-13T09:19:07.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:19:07.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434119+0000", 2024-09-13T09:19:07.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.588e-06 2024-09-13T09:19:07.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489097+0000", 2024-09-13T09:19:07.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054977824000000002 2024-09-13T09:19:07.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:07.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489146+0000", 2024-09-13T09:19:07.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8503e-05 2024-09-13T09:19:07.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489228+0000", 2024-09-13T09:19:07.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2694000000000005e-05 2024-09-13T09:19:07.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.659 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.659 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56788 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.659 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:19:07.659 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195441463000002, 2024-09-13T09:19:07.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055437562000000003, 2024-09-13T09:19:07.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:07.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56788 2024-09-13T09:19:07.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.661 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:19:07.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434122+0000", 2024-09-13T09:19:07.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434121+0000", 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434123+0000", 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2190000000000001e-06 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434126+0000", 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:19:07.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434128+0000", 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8250000000000001e-06 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489293+0000", 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055164092999999997 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:07.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489428+0000", 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013572599999999999 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489559+0000", 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013086 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56789 3.a 3:52630c54:::rbd_data.118568665621.0000000000000002:1d [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:19:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195432314000001, 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055674077000000002, 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56789 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:19:07.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434131+0000", 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434132+0000", 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.302e-06 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434136+0000", 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434139+0000", 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:19:07.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489583+0000", 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055443994000000003 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489699+0000", 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116611 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489805+0000", 2024-09-13T09:19:07.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105331 2024-09-13T09:19:07.669 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.669 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.669 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.669 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56790 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.669 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:19:07.669 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195421938999999, 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055716839999999997, 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56790 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:19:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434141+0000", 2024-09-13T09:19:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434142+0000", 2024-09-13T09:19:07.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2559999999999999e-06 2024-09-13T09:19:07.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434145+0000", 2024-09-13T09:19:07.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.96e-06 2024-09-13T09:19:07.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434148+0000", 2024-09-13T09:19:07.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6479999999999999e-06 2024-09-13T09:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489851+0000", 2024-09-13T09:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055702792000000001 2024-09-13T09:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489858+0000", 2024-09-13T09:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3900000000000004e-06 2024-09-13T09:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56791 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195413166999998, 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055740029000000003, 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56791 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.674 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434150+0000", 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434151+0000", 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434154+0000", 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0139999999999999e-06 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434157+0000", 2024-09-13T09:19:07.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6699999999999998e-06 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489884+0000", 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055727511 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489890+0000", 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7200000000000003e-06 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56792 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195404238000002, 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055760363, 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56792 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.678 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:19:07.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:19:07.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434159+0000", 2024-09-13T09:19:07.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434163+0000", 2024-09-13T09:19:07.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-13T09:19:07.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434167+0000", 2024-09-13T09:19:07.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.292e-06 2024-09-13T09:19:07.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434169+0000", 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489914+0000", 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055744415999999998 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489919+0000", 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56793 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195390717999999, 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055774049999999999, 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56793 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:19:07.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434172+0000", 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434173+0000", 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-06 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434177+0000", 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1070000000000001e-06 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434179+0000", 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.666e-06 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489941+0000", 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055762071000000003 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489946+0000", 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56794 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195381609999998, 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055791704999999997, 2024-09-13T09:19:07.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56794 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:19:07.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434181+0000", 2024-09-13T09:19:07.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434182+0000", 2024-09-13T09:19:07.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1629999999999999e-06 2024-09-13T09:19:07.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434185+0000", 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9950000000000001e-06 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434188+0000", 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489968+0000", 2024-09-13T09:19:07.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055779789000000003 2024-09-13T09:19:07.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489973+0000", 2024-09-13T09:19:07.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2800000000000003e-06 2024-09-13T09:19:07.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.692 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.692 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56795 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.692 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195372673999998, 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055809612000000001, 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56795 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:19:07.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:19:07.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434190+0000", 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434191+0000", 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.212e-06 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434194+0000", 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.078e-06 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434198+0000", 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9820000000000002e-06 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.489995+0000", 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055796363000000002 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490000+0000", 2024-09-13T09:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1839999999999998e-06 2024-09-13T09:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56796 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195362412999998, 2024-09-13T09:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055825859999999998, 2024-09-13T09:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56796 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434201+0000", 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434200+0000", 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434202+0000", 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.204e-06 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434205+0000", 2024-09-13T09:19:07.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9689999999999999e-06 2024-09-13T09:19:07.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434207+0000", 2024-09-13T09:19:07.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-13T09:19:07.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490021+0000", 2024-09-13T09:19:07.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055813929999999998 2024-09-13T09:19:07.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490027+0000", 2024-09-13T09:19:07.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.259e-06 2024-09-13T09:19:07.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.701 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56797 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.195353693999998, 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055843649000000002, 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56797 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434209+0000", 2024-09-13T09:19:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434210+0000", 2024-09-13T09:19:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2389999999999999e-06 2024-09-13T09:19:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434213+0000", 2024-09-13T09:19:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:19:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.434216+0000", 2024-09-13T09:19:07.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7800000000000001e-06 2024-09-13T09:19:07.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490048+0000", 2024-09-13T09:19:07.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055831981000000003 2024-09-13T09:19:07.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490053+0000", 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8640000000000004e-06 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56807 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.193673535999999, 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054407720999999999, 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56807 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435890+0000", 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435889+0000", 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435891+0000", 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.217e-06 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435894+0000", 2024-09-13T09:19:07.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.067e-06 2024-09-13T09:19:07.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435896+0000", 2024-09-13T09:19:07.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-13T09:19:07.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490292+0000", 2024-09-13T09:19:07.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054395840000000001 2024-09-13T09:19:07.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490297+0000", 2024-09-13T09:19:07.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:19:07.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.710 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.710 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56808 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.711 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:19:07.711 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.19366471, 2024-09-13T09:19:07.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05442524, 2024-09-13T09:19:07.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.711 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.711 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.711 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.711 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.711 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56808 2024-09-13T09:19:07.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.711 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:19:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:19:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435898+0000", 2024-09-13T09:19:07.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435900+0000", 2024-09-13T09:19:07.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:19:07.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435903+0000", 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435905+0000", 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490319+0000", 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054413396000000003 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490324+0000", 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56809 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.193654653999999, 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054442786, 2024-09-13T09:19:07.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56809 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:19:07.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435908+0000", 2024-09-13T09:19:07.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:07.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435910+0000", 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435913+0000", 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1470000000000001e-06 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435915+0000", 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6769999999999999e-06 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490346+0000", 2024-09-13T09:19:07.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054430574000000002 2024-09-13T09:19:07.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490351+0000", 2024-09-13T09:19:07.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2700000000000004e-06 2024-09-13T09:19:07.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.719 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56810 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.193645355000001, 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054459855000000001, 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56810 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:19:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435918+0000", 2024-09-13T09:19:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435917+0000", 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435919+0000", 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.457e-06 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435924+0000", 2024-09-13T09:19:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435929+0000", 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8300000000000003e-06 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490373+0000", 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054443841999999999 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490378+0000", 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8509999999999998e-06 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:56811 3.a 3.2a30c64a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.193616253999998, 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054456683999999998, 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-13T09:19:07.724 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 56811 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435947+0000", 2024-09-13T09:19:07.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:07.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:07.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435946+0000", 2024-09-13T09:19:07.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:07.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:07.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435949+0000", 2024-09-13T09:19:07.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-13T09:19:07.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:07.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435956+0000", 2024-09-13T09:19:07.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8719999999999996e-06 2024-09-13T09:19:07.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:07.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.435962+0000", 2024-09-13T09:19:07.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000004e-06 2024-09-13T09:19:07.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:07.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490398+0000", 2024-09-13T09:19:07.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054436824000000002 2024-09-13T09:19:07.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:07.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:07.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:07.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.490403+0000", 2024-09-13T09:19:07.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0710000000000002e-06 2024-09-13T09:19:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:07.729 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:07.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:19:07.732 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:07.725+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:07.832 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:07.829+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:07.906 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:07.906 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:07.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:19:07.970 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:07.965+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:08.033 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:08.029+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:08.047 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:08.047 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:08.048 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:08.048 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:08.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:19:08.135 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:08.129+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:08.235 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:08.233+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:08.237 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:19:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:19:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 116.841056349, 2024-09-13T09:19:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077478195999999999, 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392636+0000", 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392651+0000", 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:19:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392674+0000", 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.264e-05 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392691+0000", 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7203000000000001e-05 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392780+0000", 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8671999999999994e-05 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392794+0000", 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.466664+0000", 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073869913999999995 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467139+0000", 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047488599999999999 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467664+0000", 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052530600000000002 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469125+0000", 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001460432 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469847+0000", 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072197799999999996 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470006+0000", 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015955900000000001 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470116+0000", 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109836 2024-09-13T09:19:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:49389 3.d 3:b0b9272f:::rbd_data.1185e728d948.0000000000000025:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-13T09:19:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:19:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.150737573000001, 2024-09-13T09:19:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077194925999999997, 2024-09-13T09:19:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 49389 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:19:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082955+0000", 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082967+0000", 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2609000000000001e-05 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082998+0000", 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0596000000000003e-05 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083013+0000", 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4766999999999999e-05 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083136+0000", 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012295599999999999 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083360+0000", 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000224661 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.097720+0000", 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014359627 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.159975+0000", 2024-09-13T09:19:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062254728000000002 2024-09-13T09:19:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.159999+0000", 2024-09-13T09:19:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.427e-05 2024-09-13T09:19:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.160089+0000", 2024-09-13T09:19:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9895999999999994e-05 2024-09-13T09:19:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.160152+0000", 2024-09-13T09:19:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2939000000000002e-05 2024-09-13T09:19:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52426 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6209536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:19:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.428061292999999, 2024-09-13T09:19:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064408950000000006, 2024-09-13T09:19:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52426 2024-09-13T09:19:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:19:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:19:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805632+0000", 2024-09-13T09:19:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805645+0000", 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2469999999999999e-05 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805654+0000", 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7760000000000003e-06 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805661+0000", 2024-09-13T09:19:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.114e-06 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869801+0000", 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064139777999999995 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869918+0000", 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117199 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870042+0000", 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124196 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52427 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6307840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.428027362999998, 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064720860000000005, 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52427 2024-09-13T09:19:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:19:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:19:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805666+0000", 2024-09-13T09:19:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805670+0000", 2024-09-13T09:19:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2200000000000001e-06 2024-09-13T09:19:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805678+0000", 2024-09-13T09:19:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.32e-06 2024-09-13T09:19:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805685+0000", 2024-09-13T09:19:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.883e-06 2024-09-13T09:19:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870103+0000", 2024-09-13T09:19:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064417962999999995 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870241+0000", 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138362 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870388+0000", 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146732 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52428 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:19:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.428001633000001, 2024-09-13T09:19:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064990774000000001, 2024-09-13T09:19:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52428 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805692+0000", 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805695+0000", 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805704+0000", 2024-09-13T09:19:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5299999999999996e-06 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805712+0000", 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870433+0000", 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064720343999999999 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870544+0000", 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011108900000000001 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870683+0000", 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000139705 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52429 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6504448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.427961336999999, 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065238731999999994, 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52429 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:19:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:19:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805732+0000", 2024-09-13T09:19:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805736+0000", 2024-09-13T09:19:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8600000000000003e-06 2024-09-13T09:19:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805746+0000", 2024-09-13T09:19:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6860000000000001e-06 2024-09-13T09:19:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805753+0000", 2024-09-13T09:19:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:19:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870724+0000", 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064970450999999999 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870836+0000", 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112796 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870972+0000", 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135382 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52430 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6602752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.427935110999996, 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065499180000000004, 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52430 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:19:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805762+0000", 2024-09-13T09:19:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:19:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805773+0000", 2024-09-13T09:19:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1496e-05 2024-09-13T09:19:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805780+0000", 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871018+0000", 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065237396000000003 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871124+0000", 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106469 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871258+0000", 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134175 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52431 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6701056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.427908256999999, 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065755623999999999, 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52431 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:19:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805785+0000", 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805789+0000", 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2160000000000002e-06 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805797+0000", 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4870000000000002e-06 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805804+0000", 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871298+0000", 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065494259999999999 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871405+0000", 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106512 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871542+0000", 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013683599999999999 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52432 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6799360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.427843557999999, 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065976444999999995, 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52432 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:19:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805850+0000", 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805855+0000", 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8670000000000002e-06 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805867+0000", 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2597e-05 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805876+0000", 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.833e-06 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871584+0000", 2024-09-13T09:19:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065707765000000001 2024-09-13T09:19:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871690+0000", 2024-09-13T09:19:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105832 2024-09-13T09:19:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871827+0000", 2024-09-13T09:19:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013744500000000001 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52440 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7585792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.423719745, 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064276712999999999, 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52440 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809974+0000", 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809976+0000", 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8920000000000001e-06 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809981+0000", 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5170000000000001e-06 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809985+0000", 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9539999999999998e-06 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.873936+0000", 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063951470999999996 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874076+0000", 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014022199999999999 2024-09-13T09:19:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874251+0000", 2024-09-13T09:19:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017497500000000001 2024-09-13T09:19:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52441 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7684096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.423706365999998, 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064567507999999996, 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52441 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809989+0000", 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8589999999999999e-06 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809994+0000", 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809999+0000", 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874317+0000", 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064318742999999998 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874423+0000", 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106116 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874555+0000", 2024-09-13T09:19:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013196700000000001 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52442 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7782400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.423692363000001, 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064804082999999998, 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52442 2024-09-13T09:19:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:19:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810003+0000", 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8360000000000001e-06 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810009+0000", 2024-09-13T09:19:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0220000000000001e-06 2024-09-13T09:19:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810013+0000", 2024-09-13T09:19:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9879999999999998e-06 2024-09-13T09:19:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874593+0000", 2024-09-13T09:19:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064579851999999993 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874688+0000", 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4581999999999997e-05 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874806+0000", 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118108 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52443 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7880704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.423677466999997, 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065175650000000002, 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52443 2024-09-13T09:19:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:19:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:19:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:19:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810018+0000", 2024-09-13T09:19:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.798e-06 2024-09-13T09:19:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810023+0000", 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7829999999999998e-06 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810027+0000", 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8890000000000004e-06 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874886+0000", 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064859362000000004 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875029+0000", 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014311900000000001 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875192+0000", 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000163012 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52444 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7979008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.423663888, 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065471830999999994, 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52444 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810032+0000", 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8199999999999999e-06 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810036+0000", 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5510000000000001e-06 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810041+0000", 2024-09-13T09:19:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0679999999999998e-06 2024-09-13T09:19:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875256+0000", 2024-09-13T09:19:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065215765999999994 2024-09-13T09:19:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875366+0000", 2024-09-13T09:19:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109639 2024-09-13T09:19:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875502+0000", 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013630399999999999 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52445 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.423650572, 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065733503999999998, 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52445 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810043+0000", 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810045+0000", 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.813e-06 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810050+0000", 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5940000000000004e-06 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810054+0000", 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439999999999998e-06 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875543+0000", 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065489393000000007 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875658+0000", 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115302 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875777+0000", 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118776 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52446 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8175616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.423637243000002, 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065969204000000004, 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52446 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810059+0000", 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.84e-06 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810063+0000", 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4329999999999997e-06 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810067+0000", 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8800000000000001e-06 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875813+0000", 2024-09-13T09:19:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065745910000000005 2024-09-13T09:19:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875903+0000", 2024-09-13T09:19:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0665e-05 2024-09-13T09:19:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876026+0000", 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122836 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52447 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8273920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.423603002, 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066176867, 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52447 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810093+0000", 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6089999999999999e-06 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810099+0000", 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1539999999999999e-06 2024-09-13T09:19:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810104+0000", 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4630000000000003e-06 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876062+0000", 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065957672999999994 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876153+0000", 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1119000000000006e-05 2024-09-13T09:19:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876268+0000", 2024-09-13T09:19:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115431 2024-09-13T09:19:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52448 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:19:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.423586833999998, 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066534826000000005, 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52448 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810109+0000", 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0169999999999999e-06 2024-09-13T09:19:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810115+0000", 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810120+0000", 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9620000000000001e-06 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876349+0000", 2024-09-13T09:19:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066229213999999995 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876484+0000", 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013503199999999999 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876642+0000", 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158228 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52452 3.d 3:b5f40919:::rbd_data.118567e41306.0000000000000055:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.423570996000002, 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070307575999999997, 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52452 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810126+0000", 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810132+0000", 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7349999999999999e-06 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810136+0000", 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2069999999999997e-06 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876306+0000", 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066169943999999994 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876520+0000", 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021377500000000001 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.877399+0000", 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087836900000000005 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880158+0000", 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002759657 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880212+0000", 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3551999999999998e-05 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880315+0000", 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010306099999999999 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880431+0000", 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011603300000000001 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:78930 3.13 3:cc588144:::rbd_data.11853dd604c.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:59.956179+0000", 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.2775153580000005, 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067694713000000004, 2024-09-13T09:19:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78930 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956179+0000", 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956179+0000", 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956177+0000", 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956192+0000", 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5173e-05 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956223+0000", 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1056000000000002e-05 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956242+0000", 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8916999999999999e-05 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956341+0000", 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9011999999999993e-05 2024-09-13T09:19:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956470+0000", 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129603 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.969554+0000", 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013083539999999999 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.023628+0000", 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054073884000000003 2024-09-13T09:19:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.023715+0000", 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7056000000000001e-05 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.023809+0000", 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4382000000000005e-05 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.023874+0000", 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4403e-05 2024-09-13T09:19:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:08.339 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:08.340 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:19:08.341 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:08.333+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:08.441 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:08.433+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:08.477 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:08.477 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:08.477 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:08.477 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:08.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:19:08.541 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:08.537+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:08.619 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:08.619 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:08.619 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:08.620 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:08.620 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:08.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:19:08.642 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:08.641+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:08.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:19:08.759 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:08.757+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:08.767 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:08.767 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3:ed699f0f:::rbd_data.118591b29fad.000000000000008c:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 85.904197459000002, 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94878089700000001, 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.768 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:19:08.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.769 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:08.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:19:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:19:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:19:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:19:08.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810654+0000", 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94834837599999999 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810755+0000", 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100987 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810828+0000", 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2751999999999997e-05 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19465 3.10 3:0e3a5b30:::rbd_data.1185b72576e4.0000000000000062:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:08.773 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.818352722, 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97489159000000003, 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19465 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:08.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947890+0000", 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947899+0000", 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6840000000000008e-06 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947917+0000", 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7731e-05 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947925+0000", 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4189999999999996e-06 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947990+0000", 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5909999999999997e-05 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947998+0000", 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4449999999999999e-06 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.919994+0000", 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97199595900000002 2024-09-13T09:19:08.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920258+0000", 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026439500000000001 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920629+0000", 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037096299999999998 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.921797+0000", 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001167721 2024-09-13T09:19:08.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922544+0000", 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00074661200000000001 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922628+0000", 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4016000000000003e-05 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922783+0000", 2024-09-13T09:19:08.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015585499999999999 2024-09-13T09:19:08.780 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.780 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.780 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.780 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51771 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.780 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:08.780 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021616248999997, 2024-09-13T09:19:08.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083783107999999995, 2024-09-13T09:19:08.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.780 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.780 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51771 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:08.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744631+0000", 2024-09-13T09:19:08.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.943e-06 2024-09-13T09:19:08.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744639+0000", 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744646+0000", 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5860000000000001e-06 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828151+0000", 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083505732999999999 2024-09-13T09:19:08.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828286+0000", 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013457200000000001 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828411+0000", 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012564999999999999 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51772 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021593152999998, 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083939185999999999, 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51772 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744654+0000", 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2440000000000002e-06 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744662+0000", 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3659999999999992e-06 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744669+0000", 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6529999999999997e-06 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828455+0000", 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083785707000000001 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828512+0000", 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7549000000000001e-05 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828591+0000", 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8274000000000003e-05 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51773 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021569497000002, 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084139096999999996, 2024-09-13T09:19:08.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51773 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744674+0000", 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744677+0000", 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9399999999999998e-06 2024-09-13T09:19:08.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744689+0000", 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.153e-05 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744705+0000", 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5679e-05 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828658+0000", 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083953190999999996 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828734+0000", 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5784999999999994e-05 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828814+0000", 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579000000000002e-05 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51774 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021532135000001, 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084301713, 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51774 2024-09-13T09:19:08.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744723+0000", 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0832000000000001e-05 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744733+0000", 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031e-05 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744740+0000", 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0500000000000003e-06 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828867+0000", 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084127417999999995 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828932+0000", 2024-09-13T09:19:08.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4202000000000006e-05 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829014+0000", 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51775 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021498463, 2024-09-13T09:19:08.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084465871999999997, 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51775 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:08.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:08.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744749+0000", 2024-09-13T09:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5200000000000002e-06 2024-09-13T09:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744757+0000", 2024-09-13T09:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1100000000000003e-06 2024-09-13T09:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744764+0000", 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8430000000000004e-06 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829058+0000", 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084293899000000005 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829111+0000", 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3315e-05 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829212+0000", 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010079200000000001 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51776 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021474751, 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084631102999999999, 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.803 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51776 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:08.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744769+0000", 2024-09-13T09:19:08.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744772+0000", 2024-09-13T09:19:08.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.14e-06 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744781+0000", 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.276e-06 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744788+0000", 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0269999999999998e-06 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829254+0000", 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084466325999999994 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829310+0000", 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5852e-05 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829401+0000", 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1075000000000005e-05 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.807 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51777 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021419483000003, 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084901272, 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.808 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51777 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744824+0000", 2024-09-13T09:19:08.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:08.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744828+0000", 2024-09-13T09:19:08.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:19:08.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744840+0000", 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1997000000000001e-05 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744847+0000", 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0659999999999998e-06 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829460+0000", 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084613387999999998 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829555+0000", 2024-09-13T09:19:08.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4999000000000003e-05 2024-09-13T09:19:08.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829726+0000", 2024-09-13T09:19:08.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017100899999999999 2024-09-13T09:19:08.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.812 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51778 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021367038999998, 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085026334999999995, 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51778 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.813 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744877+0000", 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744881+0000", 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5999999999999998e-06 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744892+0000", 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1732999999999999e-05 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744899+0000", 2024-09-13T09:19:08.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0759999999999997e-06 2024-09-13T09:19:08.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829772+0000", 2024-09-13T09:19:08.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084872633000000003 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829826+0000", 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4330000000000003e-05 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829904+0000", 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7558999999999994e-05 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51779 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021338470000003, 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085200608999999997, 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51779 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744905+0000", 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744909+0000", 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3500000000000001e-06 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744917+0000", 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3129999999999999e-06 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744924+0000", 2024-09-13T09:19:08.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6059999999999999e-06 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829976+0000", 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085051775999999996 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830030+0000", 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4404000000000003e-05 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830107+0000", 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6796000000000001e-05 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51780 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021314877000002, 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085367775000000007, 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.822 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51780 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744929+0000", 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744932+0000", 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744940+0000", 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1130000000000001e-06 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744947+0000", 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830148+0000", 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085201099000000002 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830205+0000", 2024-09-13T09:19:08.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6623999999999999e-05 2024-09-13T09:19:08.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830297+0000", 2024-09-13T09:19:08.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2693000000000002e-05 2024-09-13T09:19:08.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.826 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51781 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021291841, 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085544856000000002, 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51781 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.827 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744952+0000", 2024-09-13T09:19:08.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744955+0000", 2024-09-13T09:19:08.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0960000000000001e-06 2024-09-13T09:19:08.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745048+0000", 2024-09-13T09:19:08.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2806999999999994e-05 2024-09-13T09:19:08.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745058+0000", 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0132999999999999e-05 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830338+0000", 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085279761999999995 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830420+0000", 2024-09-13T09:19:08.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1551000000000006e-05 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830498+0000", 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8103999999999996e-05 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51782 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1794048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:08.831 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021179918999998, 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085605889000000004, 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51782 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:08.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745064+0000", 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745068+0000", 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.523e-06 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745076+0000", 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.354e-06 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745091+0000", 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5274999999999999e-05 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830538+0000", 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085447238999999994 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830591+0000", 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2481000000000001e-05 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830671+0000", 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9716999999999996e-05 2024-09-13T09:19:08.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51783 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1892352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021147272999997, 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085732662000000001, 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51783 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745100+0000", 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745108+0000", 2024-09-13T09:19:08.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4360000000000002e-06 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745115+0000", 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9269999999999999e-06 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830707+0000", 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085591133999999999 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830758+0000", 2024-09-13T09:19:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1180000000000001e-05 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830830+0000", 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2268999999999995e-05 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51784 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:08.840 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.021115305000002, 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085902834999999997, 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51784 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:08.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:08.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:08.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745132+0000", 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.36e-06 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745143+0000", 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745150+0000", 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0400000000000004e-06 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830870+0000", 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085720086000000001 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830955+0000", 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5321000000000005e-05 2024-09-13T09:19:08.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831032+0000", 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6811999999999996e-05 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51785 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.020560410000002, 2024-09-13T09:19:08.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085542517999999998, 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51785 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:08.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745160+0000", 2024-09-13T09:19:08.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994755 2024-09-13T09:19:08.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745697+0000", 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00053713000000000003 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745731+0000", 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3767999999999999e-05 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745754+0000", 2024-09-13T09:19:08.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3748999999999999e-05 2024-09-13T09:19:08.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831075+0000", 2024-09-13T09:19:08.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085321067 2024-09-13T09:19:08.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831130+0000", 2024-09-13T09:19:08.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4156999999999997e-05 2024-09-13T09:19:08.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831227+0000", 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7093e-05 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51789 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.017867654, 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083633549000000001, 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.850 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51789 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748376+0000", 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748380+0000", 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7529999999999999e-06 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748392+0000", 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2376e-05 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748399+0000", 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0199999999999997e-06 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831866+0000", 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083466636999999996 2024-09-13T09:19:08.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831933+0000", 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7144000000000006e-05 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832010+0000", 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7352000000000001e-05 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51790 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.017839088000002, 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083812485000000006, 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51790 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.855 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748405+0000", 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748408+0000", 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3859999999999999e-06 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748417+0000", 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748424+0000", 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832055+0000", 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083631469 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832129+0000", 2024-09-13T09:19:08.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4240999999999996e-05 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832218+0000", 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8562999999999999e-05 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51791 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2678784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.017814669000003, 2024-09-13T09:19:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083959632000000006, 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51791 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748429+0000", 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:08.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748432+0000", 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0800000000000002e-06 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748441+0000", 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9120000000000004e-06 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748470+0000", 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569e-05 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832260+0000", 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083790586 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832317+0000", 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6054999999999998e-05 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832390+0000", 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3033000000000004e-05 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.863 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:08.864 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:08.864 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:08.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:19:08.866 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:08.861+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:08.966 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:08.961+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:09.030 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:09.031 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:09.031 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:09.031 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:09.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:19:09.067 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:09.061+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:09.164 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:09.164 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:09.164 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:09.164 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:09.165 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:09.168 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:09.165+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:09.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:19:09.269 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:09.265+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:09.350 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.350 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:19:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 63.273257155000003, 2024-09-13T09:19:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:19:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:19:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.351 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:19:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:19:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:19:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:19:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:19:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:19:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:19:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:19:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:19:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:19:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:19:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:19:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:09.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:19:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:19:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:19:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:19:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:19:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:19:09.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56102 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5439488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.930416098000002, 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060495140000000003, 2024-09-13T09:19:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56102 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416845+0000", 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416853+0000", 2024-09-13T09:19:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416862+0000", 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1959999999999997e-06 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416868+0000", 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9719999999999997e-06 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477088+0000", 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060219898000000001 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477154+0000", 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5605000000000006e-05 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477341+0000", 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018687599999999999 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56103 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5537792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.930388034000003, 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060576972999999999, 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56103 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416873+0000", 2024-09-13T09:19:09.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416876+0000", 2024-09-13T09:19:09.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.993e-06 2024-09-13T09:19:09.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416883+0000", 2024-09-13T09:19:09.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.477e-06 2024-09-13T09:19:09.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416888+0000", 2024-09-13T09:19:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2580000000000003e-06 2024-09-13T09:19:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477357+0000", 2024-09-13T09:19:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060469055000000001 2024-09-13T09:19:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477396+0000", 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9402e-05 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477451+0000", 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4262000000000003e-05 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.367 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56104 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5636096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.368 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:09.368 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.930369542000001, 2024-09-13T09:19:09.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060778881999999999, 2024-09-13T09:19:09.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.368 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.368 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.368 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.368 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56104 2024-09-13T09:19:09.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416894+0000", 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4049999999999998e-06 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416900+0000", 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.959e-06 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416905+0000", 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477465+0000", 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060559460000000002 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477521+0000", 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5813000000000002e-05 2024-09-13T09:19:09.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477671+0000", 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150366 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56105 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5734400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.930325330999999, 2024-09-13T09:19:09.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060898832, 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56105 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:09.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:09.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:09.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416939+0000", 2024-09-13T09:19:09.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3550000000000001e-06 2024-09-13T09:19:09.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416947+0000", 2024-09-13T09:19:09.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0949999999999996e-06 2024-09-13T09:19:09.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416953+0000", 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6790000000000002e-06 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477708+0000", 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060755454 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477760+0000", 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1589999999999999e-05 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477835+0000", 2024-09-13T09:19:09.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5394000000000004e-05 2024-09-13T09:19:09.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.377 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.377 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56106 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5832704~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.377 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:09.377 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.930304014000001, 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061142999000000003, 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56106 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:09.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416957+0000", 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416960+0000", 2024-09-13T09:19:09.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6979999999999998e-06 2024-09-13T09:19:09.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416967+0000", 2024-09-13T09:19:09.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6919999999999996e-06 2024-09-13T09:19:09.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416973+0000", 2024-09-13T09:19:09.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5679999999999996e-06 2024-09-13T09:19:09.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477867+0000", 2024-09-13T09:19:09.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060894082000000002 2024-09-13T09:19:09.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477924+0000", 2024-09-13T09:19:09.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7015999999999998e-05 2024-09-13T09:19:09.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478101+0000", 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017648300000000001 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56107 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5931008~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.929975400000004, 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060956639999999999, 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.382 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56107 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:09.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416977+0000", 2024-09-13T09:19:09.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996905 2024-09-13T09:19:09.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417291+0000", 2024-09-13T09:19:09.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031392799999999999 2024-09-13T09:19:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417311+0000", 2024-09-13T09:19:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0343000000000002e-05 2024-09-13T09:19:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417329+0000", 2024-09-13T09:19:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7612e-05 2024-09-13T09:19:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478123+0000", 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060794421000000001 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478174+0000", 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0614e-05 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478243+0000", 2024-09-13T09:19:09.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9226000000000004e-05 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56108 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 6029312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.929915915999999, 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061163618000000003, 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.387 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56108 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417345+0000", 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417349+0000", 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7419999999999999e-06 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417367+0000", 2024-09-13T09:19:09.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7714999999999999e-05 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417382+0000", 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5396999999999998e-05 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478270+0000", 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060888022999999999 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478322+0000", 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2234999999999997e-05 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478509+0000", 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187109 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56121 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7307264~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.391 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.925932422999999, 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060455939, 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56121 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:09.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421332+0000", 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421358+0000", 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6427999999999998e-05 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421374+0000", 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6137000000000002e-05 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481604+0000", 2024-09-13T09:19:09.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060229528999999997 2024-09-13T09:19:09.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481680+0000", 2024-09-13T09:19:09.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6089999999999998e-05 2024-09-13T09:19:09.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481785+0000", 2024-09-13T09:19:09.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105259 2024-09-13T09:19:09.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.396 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.396 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56122 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7405568~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.396 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:09.396 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.925858920000003, 2024-09-13T09:19:09.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060604366999999999, 2024-09-13T09:19:09.396 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.396 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.396 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56122 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:09.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421402+0000", 2024-09-13T09:19:09.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421406+0000", 2024-09-13T09:19:09.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.337e-06 2024-09-13T09:19:09.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421415+0000", 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421434+0000", 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8617000000000001e-05 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481827+0000", 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060393883000000002 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481910+0000", 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2657999999999994e-05 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482007+0000", 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7057000000000003e-05 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.400 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56123 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7503872~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.925822382, 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060838125999999999, 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.401 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56123 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421450+0000", 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1544e-05 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421463+0000", 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421468+0000", 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6250000000000004e-06 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482066+0000", 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060597713999999997 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482163+0000", 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7112000000000001e-05 2024-09-13T09:19:09.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482277+0000", 2024-09-13T09:19:09.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114274 2024-09-13T09:19:09.405 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.405 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.405 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.405 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56124 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7602176~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.405 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:09.405 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.925788427000001, 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061054929000000001, 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56124 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421476+0000", 2024-09-13T09:19:09.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2059999999999999e-06 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421484+0000", 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7889999999999995e-06 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421489+0000", 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482321+0000", 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060831477000000002 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482403+0000", 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2776999999999997e-05 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482528+0000", 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012491600000000001 2024-09-13T09:19:09.409 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56125 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7700480~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.925768130000002, 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061273721000000003, 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.410 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56125 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421493+0000", 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421496+0000", 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.666e-06 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421503+0000", 2024-09-13T09:19:09.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421508+0000", 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0919999999999997e-06 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482575+0000", 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061066339999999997 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482659+0000", 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4158000000000003e-05 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482767+0000", 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010849 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56126 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7798784~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.414 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.925748763999998, 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061489349999999998, 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56126 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:09.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421531+0000", 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8218000000000001e-05 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421538+0000", 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.345e-06 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421544+0000", 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482813+0000", 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061268959999999997 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482900+0000", 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6858000000000001e-05 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483002+0000", 2024-09-13T09:19:09.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102599 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56127 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7897088~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.925713023, 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061729777, 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56127 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421548+0000", 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421551+0000", 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421559+0000", 2024-09-13T09:19:09.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4499999999999998e-06 2024-09-13T09:19:09.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421564+0000", 2024-09-13T09:19:09.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:19:09.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483067+0000", 2024-09-13T09:19:09.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061503073999999998 2024-09-13T09:19:09.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483153+0000", 2024-09-13T09:19:09.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:09.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483278+0000", 2024-09-13T09:19:09.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012522499999999999 2024-09-13T09:19:09.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.423 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56319 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1613824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.919955277, 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060599663999999998, 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56319 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.424 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:09.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:09.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427309+0000", 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.672e-06 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427316+0000", 2024-09-13T09:19:09.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6850000000000001e-06 2024-09-13T09:19:09.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427322+0000", 2024-09-13T09:19:09.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:19:09.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487756+0000", 2024-09-13T09:19:09.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060434817000000002 2024-09-13T09:19:09.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487825+0000", 2024-09-13T09:19:09.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8553000000000006e-05 2024-09-13T09:19:09.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487906+0000", 2024-09-13T09:19:09.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1067000000000002e-05 2024-09-13T09:19:09.428 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.428 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.428 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56320 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1712128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.919935463000002, 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060803930999999999, 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56320 2024-09-13T09:19:09.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.430 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:09.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:09.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427328+0000", 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427379+0000", 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0343999999999998e-05 2024-09-13T09:19:09.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427385+0000", 2024-09-13T09:19:09.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3969999999999999e-06 2024-09-13T09:19:09.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487941+0000", 2024-09-13T09:19:09.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060555805999999997 2024-09-13T09:19:09.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488034+0000", 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488130+0000", 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6348999999999995e-05 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.433 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56321 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1810432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.434 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:09.434 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.919870525999997, 2024-09-13T09:19:09.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060999937999999997, 2024-09-13T09:19:09.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.434 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56321 2024-09-13T09:19:09.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.434 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:09.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:09.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:09.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:09.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427394+0000", 2024-09-13T09:19:09.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.721e-06 2024-09-13T09:19:09.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427403+0000", 2024-09-13T09:19:09.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7110000000000001e-06 2024-09-13T09:19:09.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427409+0000", 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9710000000000001e-06 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488183+0000", 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060774003 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488291+0000", 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010786899999999999 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488391+0000", 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100142 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56322 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1908736~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.438 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.919524410999998, 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060864109999999999, 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56322 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427433+0000", 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996958 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427742+0000", 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000309084 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427754+0000", 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1354e-05 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427761+0000", 2024-09-13T09:19:09.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8059999999999995e-06 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488429+0000", 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060667139000000002 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488496+0000", 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7595000000000006e-05 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488601+0000", 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105257 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56323 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 2007040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.918918716, 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060546618000000003, 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:09.443 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56323 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:09.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427767+0000", 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994245 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428348+0000", 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00058032600000000004 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:09.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428360+0000", 2024-09-13T09:19:09.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2103000000000001e-05 2024-09-13T09:19:09.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:09.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428367+0000", 2024-09-13T09:19:09.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1949999999999997e-06 2024-09-13T09:19:09.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:09.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488650+0000", 2024-09-13T09:19:09.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060282626999999998 2024-09-13T09:19:09.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488745+0000", 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5365999999999995e-05 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488890+0000", 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000144506 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:09.447 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:09.448 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:09.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:19:09.450 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:09.445+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:09.503 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:19:09.550 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:09.545+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:09.624 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:09.625 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:09.625 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:09.625 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:09.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:19:09.651 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:09.649+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:09.697 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:19:09.697 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:19:09.697 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T09:19:09.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:19:09.698 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:09.698 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:09.719 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:19:09.720 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:09.739 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:09.753 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:09.745+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:09.759 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:09.770 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:09.770 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:09.781 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:09.781 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:09.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:19:09.805 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:09.805 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:09.835 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:09.854 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:09.849+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:09.873 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:09.873 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:09.944 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:09.944 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:09.955 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:09.949+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:09.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:19:10.016 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:19:10.016 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:10.016 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 107, 2024-09-13T09:19:10.016 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "f0889cb1-03d6-4b81-88c9-6141846c6033", 2024-09-13T09:19:10.016 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-13T09:16:56.964982+0000", 2024-09-13T09:19:10.016 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-13T09:19:09.707372+0000", 2024-09-13T09:19:10.016 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-13T09:17:08.368378+0000", 2024-09-13T09:19:10.016 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-13T09:19:04.653692+0000", 2024-09-13T09:19:10.016 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5804032, 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: "nodeep-scrub", 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:19:10.017 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-13T09:17:10.140920+0000", 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-13T09:19:10.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-13T09:19:10.019 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-13T09:19:10.020 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-13T09:19:10.020 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-13T09:19:10.020 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-13T09:19:10.020 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-13T09:19:10.020 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-13T09:19:10.020 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-13T09:19:10.020 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-13T09:19:10.020 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-13T09:19:10.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.020 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "17", 2024-09-13T09:19:10.020 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-13T09:19:10.020 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-13T09:19:10.021 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-13T09:19:10.022 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-13T09:19:10.022 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-13T09:19:10.022 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-13T09:19:10.022 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:19:10.022 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:19:10.022 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:19:10.022 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-13T09:19:10.022 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-13T09:19:10.022 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-13T09:19:10.022 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-13T09:19:10.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-13T09:19:10.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-13T09:19:10.023 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-13T09:19:10.024 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-13T09:19:10.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.024 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-13T09:19:10.024 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-13T09:19:10.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.024 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-13T09:19:10.024 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-13T09:19:10.024 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-13T09:19:10.025 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-13T09:19:10.025 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-13T09:19:10.025 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-13T09:19:10.025 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.98870849609375, 2024-09-13T09:19:10.025 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-13T09:19:10.025 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.99000000953674316 2024-09-13T09:19:10.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.025 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-13T09:19:10.025 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-13T09:19:10.025 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-13T09:17:15.775544+0000", 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:19:10.026 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "21", 2024-09-13T09:19:10.027 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 21, 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-13T09:19:10.028 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-13T09:19:10.029 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-13T09:19:10.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2.9900000095367432, 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2.9900000095367432, 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2.630000114440918, 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2.630000114440918, 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.98870849609375, 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.99000000953674316 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-13T09:19:10.031 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-13T09:17:25.804931+0000", 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:19:10.032 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.033 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "107", 2024-09-13T09:19:10.034 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-13T09:19:10.034 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:19:10.034 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:19:10.034 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-13T09:19:10.034 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:19:10.034 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 48, 2024-09-13T09:19:10.034 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 107, 2024-09-13T09:19:10.034 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-13T09:19:10.034 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-13T09:19:10.034 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-13T09:19:10.034 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-13T09:19:10.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-13T09:19:10.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-13T09:19:10.036 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-13T09:19:10.037 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-13T09:19:10.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.037 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-13T09:19:10.037 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.4900000095367432, 2024-09-13T09:19:10.037 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.4900000095367432, 2024-09-13T09:19:10.037 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T09:19:10.037 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.309999942779541, 2024-09-13T09:19:10.037 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.309999942779541, 2024-09-13T09:19:10.037 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.98870849609375, 2024-09-13T09:19:10.037 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-13T09:19:10.037 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.99000000953674316 2024-09-13T09:19:10.037 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "786bdfc4-cc25-4167-aa5b-3737cfeb4ac1", 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:10.038 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.039 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-13T09:19:10.040 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:10.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.040 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:10.040 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.040 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-13T09:19:10.040 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:10.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.041 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.041 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-13T09:19:10.041 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:10.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.041 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.041 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:10.041 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.041 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.041 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-13T09:19:10.041 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-13T09:19:10.042 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/3549111487", 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/3549111487", 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/3549111487", 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/3549111487", 2024-09-13T09:19:10.043 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "ca6189ab-90dd-4e9d-a85f-092186153b5a", 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:10.044 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:10.045 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:10.045 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 101, 2024-09-13T09:19:10.045 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:10.045 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:10.045 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:10.045 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.045 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-13T09:19:10.045 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-13T09:19:10.046 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-13T09:19:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-13T09:19:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.049 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-13T09:19:10.049 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:10.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.049 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-13T09:19:10.049 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:10.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.049 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.049 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6810/1470371899", 2024-09-13T09:19:10.049 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6815/1470371899", 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1470371899", 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6820/1470371899", 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a", 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:10.050 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:10.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:10.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-13T09:19:10.053 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.054 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/2485123706", 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/2485123706", 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/2485123706", 2024-09-13T09:19:10.055 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/2485123706", 2024-09-13T09:19:10.056 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:10.056 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:10.056 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:10.056 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.056 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-13T09:19:10.056 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218", 2024-09-13T09:19:10.056 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:10.056 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:10.056 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:10.056 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:10.057 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:10.057 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:10.057 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:10.057 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2024-09-13T09:19:10.057 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:10.057 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:10.057 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:10.057 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.057 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-13T09:19:10.057 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-13T09:19:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-13T09:19:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.061 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/1009890408", 2024-09-13T09:19:10.062 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6814/1009890408", 2024-09-13T09:19:10.062 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6821/1009890408", 2024-09-13T09:19:10.062 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6818/1009890408", 2024-09-13T09:19:10.062 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:10.062 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:10.062 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:10.062 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.062 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-13T09:19:10.062 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39f42e87-29dd-42cc-9ef6-e0074f1794e4", 2024-09-13T09:19:10.062 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.063 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-13T09:19:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-13T09:19:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:10.065 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.066 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6823/2723023689", 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6825/2723023689", 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/2723023689", 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6828/2723023689", 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "cf8dd701-9543-4bdd-a6f6-0b56210f00c2", 2024-09-13T09:19:10.068 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.071 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-13T09:19:10.071 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:10.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.071 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-13T09:19:10.071 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:10.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.071 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:10.072 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.072 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-13T09:19:10.072 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:10.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.072 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-13T09:19:10.072 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:10.072 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.072 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.073 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:10.073 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.073 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-13T09:19:10.073 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:10.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.073 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-13T09:19:10.074 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:10.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.074 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.074 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6809/2955019640", 2024-09-13T09:19:10.074 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6812/2955019640", 2024-09-13T09:19:10.074 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6819/2955019640", 2024-09-13T09:19:10.074 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6815/2955019640", 2024-09-13T09:19:10.074 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:10.075 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:10.075 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:10.075 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.075 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-13T09:19:10.075 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "dd080b88-3292-4a34-8dbd-645683d8fe14", 2024-09-13T09:19:10.075 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:10.075 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 0, 2024-09-13T09:19:10.075 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0, 2024-09-13T09:19:10.075 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.90966796875, 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 75, 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:10.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.077 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-13T09:19:10.077 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:10.077 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.077 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.077 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:10.077 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.078 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-13T09:19:10.078 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:10.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.078 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-13T09:19:10.078 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:10.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.078 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.079 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.080 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:10.080 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.080 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-13T09:19:10.080 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:10.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.080 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-13T09:19:10.080 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6813/916160151", 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6818/916160151", 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6829/916160151", 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6822/916160151", 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c6bca54b-e982-4d6a-9ae9-057ee3881a5b", 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 101, 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:10.082 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:10.083 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:10.084 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.085 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6801/3477167689", 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6803/3477167689", 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6807/3477167689", 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6805/3477167689", 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.087 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.247113+0000", 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:10.088 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.166872+0000", 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.451769+0000", 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:10.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.458130+0000", 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:10.090 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.466129+0000", 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:10.091 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.444828+0000", 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 65536, 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.425145+0000", 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:10.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.484371+0000", 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-09-13T09:19:10.093 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.6", 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 0, 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 2 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-13T09:19:10.094 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-13T09:19:10.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-13T09:19:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-13T09:19:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 37, 2024-09-13T09:19:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 3 2024-09-13T09:19:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [ 2024-09-13T09:19:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.096 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-13T09:19:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-13T09:19:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 37, 2024-09-13T09:19:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-13T09:19:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-09-13T09:19:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-13T09:19:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-13T09:19:10.097 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-13T09:19:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-13T09:19:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-13T09:19:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:19:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:19:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-13T09:19:10.098 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:10.098 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:10.100 INFO:tasks.thrashosds.thrasher:j is {'epoch': 107, 'fsid': 'f0889cb1-03d6-4b81-88c9-6141846c6033', 'created': '2024-09-13T09:16:56.964982+0000', 'modified': '2024-09-13T09:19:09.707372+0000', 'last_up_change': '2024-09-13T09:17:08.368378+0000', 'last_in_change': '2024-09-13T09:19:04.653692+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:17:10.140920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.98870849609375, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.9900000095367432}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:17:15.775544+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.98870849609375, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.9900000095367432}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:17:25.804931+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 0.98870849609375, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.9900000095367432}}], 'osds': [{'osd': 0, 'uuid': '786bdfc4-cc25-4167-aa5b-3737cfeb4ac1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 105, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 3549111487}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 3549111487}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 3549111487}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 3549111487}]}, 'public_addr': '172.21.15.32:6801/3549111487', 'cluster_addr': '172.21.15.32:6803/3549111487', 'heartbeat_back_addr': '172.21.15.32:6807/3549111487', 'heartbeat_front_addr': '172.21.15.32:6805/3549111487', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'ca6189ab-90dd-4e9d-a85f-092186153b5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6810', 'nonce': 1470371899}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 1470371899}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1470371899}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6817', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6820', 'nonce': 1470371899}]}, 'public_addr': '172.21.15.32:6810/1470371899', 'cluster_addr': '172.21.15.32:6815/1470371899', 'heartbeat_back_addr': '172.21.15.32:6823/1470371899', 'heartbeat_front_addr': '172.21.15.32:6820/1470371899', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 2485123706}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 2485123706}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 2485123706}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 2485123706}]}, 'public_addr': '172.21.15.32:6825/2485123706', 'cluster_addr': '172.21.15.32:6827/2485123706', 'heartbeat_back_addr': '172.21.15.32:6831/2485123706', 'heartbeat_front_addr': '172.21.15.32:6829/2485123706', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 105, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6809', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 1009890408}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6813', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6814', 'nonce': 1009890408}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6819', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1009890408}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6818', 'nonce': 1009890408}]}, 'public_addr': '172.21.15.32:6811/1009890408', 'cluster_addr': '172.21.15.32:6814/1009890408', 'heartbeat_back_addr': '172.21.15.32:6821/1009890408', 'heartbeat_front_addr': '172.21.15.32:6818/1009890408', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39f42e87-29dd-42cc-9ef6-e0074f1794e4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 105, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6823', 'nonce': 2723023689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6824', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 2723023689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 2723023689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 2723023689}]}, 'public_addr': '172.21.15.46:6823/2723023689', 'cluster_addr': '172.21.15.46:6825/2723023689', 'heartbeat_back_addr': '172.21.15.46:6831/2723023689', 'heartbeat_front_addr': '172.21.15.46:6828/2723023689', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'cf8dd701-9543-4bdd-a6f6-0b56210f00c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6809', 'nonce': 2955019640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6811', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6812', 'nonce': 2955019640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 2955019640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6814', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 2955019640}]}, 'public_addr': '172.21.15.46:6809/2955019640', 'cluster_addr': '172.21.15.46:6812/2955019640', 'heartbeat_back_addr': '172.21.15.46:6819/2955019640', 'heartbeat_front_addr': '172.21.15.46:6815/2955019640', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'dd080b88-3292-4a34-8dbd-645683d8fe14', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.90966796875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6810', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6813', 'nonce': 916160151}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 916160151}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 916160151}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 916160151}]}, 'public_addr': '172.21.15.46:6813/916160151', 'cluster_addr': '172.21.15.46:6818/916160151', 'heartbeat_back_addr': '172.21.15.46:6829/916160151', 'heartbeat_front_addr': '172.21.15.46:6822/916160151', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c6bca54b-e982-4d6a-9ae9-057ee3881a5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6801', 'nonce': 3477167689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6802', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 3477167689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6806', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 3477167689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6805', 'nonce': 3477167689}]}, 'public_addr': '172.21.15.46:6801/3477167689', 'cluster_addr': '172.21.15.46:6803/3477167689', 'heartbeat_back_addr': '172.21.15.46:6807/3477167689', 'heartbeat_front_addr': '172.21.15.46:6805/3477167689', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.247113+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.166872+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.451769+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.458130+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.466129+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.444828+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.425145+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.484371+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.6', 'mappings': [{'from': 0, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 37, 'length': 3}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 37, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:19:10.100 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.6 2024-09-13T09:19:10.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.6 2024-09-13T09:19:10.102 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:10.102 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:10.102 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:19:10.102 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:10.097+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:10.200 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:10.193+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:10.267 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:10.267 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:10.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:19:10.301 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:10.293+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:10.391 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:10.391 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:10.402 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:10.401+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:10.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:19:10.503 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:10.497+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:10.538 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:10.538 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:10.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:19:10.603 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:10.597+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:10.672 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:10.672 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:10.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:19:10.704 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:10.701+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:10.736 INFO:teuthology.orchestra.run.smithi032.stderr:clear 3.6 pg_upmap_items mapping 2024-09-13T09:19:10.807 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:10.805+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:10.858 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:10.859 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:10.873 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:19:10.905 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:10.901+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:10.998 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:10.999 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:11.006 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:11.005+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:11.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:19:11.108 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:11.105+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:11.134 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:11.134 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:11.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:19:11.208 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:11.201+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:11.275 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:11.275 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:11.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:19:11.334 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:11.329+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:11.410 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:11.405+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:11.461 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:11.461 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:11.476 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:19:11.521 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:11.517+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:11.605 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:11.605 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:11.612 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:11.609+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:11.619 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:19:11.712 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:11.705+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:11.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:19:11.790 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:11.790 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:11.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:19:11.813 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:11.809+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:11.899 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:19:11.914 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:11.909+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:11.979 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:11.979 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:11.993 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:19:12.015 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:12.009+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:12.038 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:12.039 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:12.054 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:12.071 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:12.071 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:12.088 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:12.105 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:12.106 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:12.116 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:12.109+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:12.125 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:12.145 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:12.166 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:12.166 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:12.167 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:12.167 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:12.167 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:12.167 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:12.167 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:12.181 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:19:12.226 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:12.221+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:12.318 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:12.313+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:12.348 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:19:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:19:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 120.952541813, 2024-09-13T09:19:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077478195999999999, 2024-09-13T09:19:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4185", 2024-09-13T09:19:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/2294319258", 2024-09-13T09:19:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1 2024-09-13T09:19:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:19:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392638+0000", 2024-09-13T09:19:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392636+0000", 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392651+0000", 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392674+0000", 2024-09-13T09:19:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.264e-05 2024-09-13T09:19:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392691+0000", 2024-09-13T09:19:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7203000000000001e-05 2024-09-13T09:19:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392780+0000", 2024-09-13T09:19:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8671999999999994e-05 2024-09-13T09:19:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.392794+0000", 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.466664+0000", 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073869913999999995 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467139+0000", 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047488599999999999 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.467664+0000", 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052530600000000002 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469125+0000", 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001460432 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.469847+0000", 2024-09-13T09:19:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072197799999999996 2024-09-13T09:19:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470006+0000", 2024-09-13T09:19:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015955900000000001 2024-09-13T09:19:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:11.470116+0000", 2024-09-13T09:19:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109836 2024-09-13T09:19:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:49389 3.d 3:b0b9272f:::rbd_data.1185e728d948.0000000000000025:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-13T09:19:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:19:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.262223036999998, 2024-09-13T09:19:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.077194925999999997, 2024-09-13T09:19:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 49389 2024-09-13T09:19:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:19:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082957+0000", 2024-09-13T09:19:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082955+0000", 2024-09-13T09:19:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082967+0000", 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2609000000000001e-05 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.082998+0000", 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0596000000000003e-05 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083013+0000", 2024-09-13T09:19:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4766999999999999e-05 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083136+0000", 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012295599999999999 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.083360+0000", 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000224661 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.097720+0000", 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014359627 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.159975+0000", 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062254728000000002 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.159999+0000", 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.427e-05 2024-09-13T09:19:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.160089+0000", 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9895999999999994e-05 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:25.160152+0000", 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2939000000000002e-05 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52426 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6209536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.539546756999997, 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064408950000000006, 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52426 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805633+0000", 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805632+0000", 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805645+0000", 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2469999999999999e-05 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805654+0000", 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7760000000000003e-06 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805661+0000", 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.114e-06 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869801+0000", 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064139777999999995 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.869918+0000", 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117199 2024-09-13T09:19:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870042+0000", 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124196 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52427 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6307840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.539512827000003, 2024-09-13T09:19:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064720860000000005, 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52427 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805667+0000", 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805666+0000", 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805670+0000", 2024-09-13T09:19:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2200000000000001e-06 2024-09-13T09:19:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805678+0000", 2024-09-13T09:19:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.32e-06 2024-09-13T09:19:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805685+0000", 2024-09-13T09:19:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.883e-06 2024-09-13T09:19:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870103+0000", 2024-09-13T09:19:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064417962999999995 2024-09-13T09:19:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870241+0000", 2024-09-13T09:19:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138362 2024-09-13T09:19:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870388+0000", 2024-09-13T09:19:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146732 2024-09-13T09:19:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52428 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:19:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.539487096999999, 2024-09-13T09:19:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064990774000000001, 2024-09-13T09:19:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52428 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805693+0000", 2024-09-13T09:19:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805692+0000", 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805695+0000", 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805704+0000", 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5299999999999996e-06 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805712+0000", 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870433+0000", 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064720343999999999 2024-09-13T09:19:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870544+0000", 2024-09-13T09:19:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011108900000000001 2024-09-13T09:19:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870683+0000", 2024-09-13T09:19:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000139705 2024-09-13T09:19:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52429 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6504448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:19:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.539446800999997, 2024-09-13T09:19:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065238731999999994, 2024-09-13T09:19:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52429 2024-09-13T09:19:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:19:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805733+0000", 2024-09-13T09:19:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805732+0000", 2024-09-13T09:19:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805736+0000", 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8600000000000003e-06 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805746+0000", 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6860000000000001e-06 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805753+0000", 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870724+0000", 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064970450999999999 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870836+0000", 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112796 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.870972+0000", 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135382 2024-09-13T09:19:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52430 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6602752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.539420575000001, 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065499180000000004, 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52430 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805759+0000", 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805762+0000", 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805773+0000", 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1496e-05 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805780+0000", 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871018+0000", 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065237396000000003 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871124+0000", 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106469 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871258+0000", 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134175 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52431 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6701056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.539393721000003, 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065755623999999999, 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52431 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805786+0000", 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805785+0000", 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805789+0000", 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2160000000000002e-06 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805797+0000", 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4870000000000002e-06 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805804+0000", 2024-09-13T09:19:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871298+0000", 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065494259999999999 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871405+0000", 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106512 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871542+0000", 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013683599999999999 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52432 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 6799360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.539329021999997, 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065976444999999995, 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52432 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805851+0000", 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805850+0000", 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805855+0000", 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8670000000000002e-06 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805867+0000", 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2597e-05 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.805876+0000", 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.833e-06 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871584+0000", 2024-09-13T09:19:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065707765000000001 2024-09-13T09:19:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871690+0000", 2024-09-13T09:19:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105832 2024-09-13T09:19:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.871827+0000", 2024-09-13T09:19:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013744500000000001 2024-09-13T09:19:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52440 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7585792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.535205208999997, 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064276712999999999, 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52440 2024-09-13T09:19:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:19:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809975+0000", 2024-09-13T09:19:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809974+0000", 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809976+0000", 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8920000000000001e-06 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809981+0000", 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5170000000000001e-06 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809985+0000", 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9539999999999998e-06 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.873936+0000", 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063951470999999996 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874076+0000", 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014022199999999999 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874251+0000", 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017497500000000001 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52441 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7684096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.535191830000002, 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064567507999999996, 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52441 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:19:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:19:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809988+0000", 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809989+0000", 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8589999999999999e-06 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809994+0000", 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.809999+0000", 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874317+0000", 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064318742999999998 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874423+0000", 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106116 2024-09-13T09:19:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874555+0000", 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013196700000000001 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52442 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7782400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.535177826999998, 2024-09-13T09:19:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064804082999999998, 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52442 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810002+0000", 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810003+0000", 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8360000000000001e-06 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810009+0000", 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0220000000000001e-06 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810013+0000", 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9879999999999998e-06 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874593+0000", 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064579851999999993 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874688+0000", 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4581999999999997e-05 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874806+0000", 2024-09-13T09:19:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118108 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52443 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7880704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.535162931000002, 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065175650000000002, 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52443 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:19:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810017+0000", 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810018+0000", 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.798e-06 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810023+0000", 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7829999999999998e-06 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810027+0000", 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8890000000000004e-06 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.874886+0000", 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064859362000000004 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875029+0000", 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014311900000000001 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875192+0000", 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000163012 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52444 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 7979008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.535149351999998, 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065471830999999994, 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52444 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810030+0000", 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810032+0000", 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8199999999999999e-06 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810036+0000", 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5510000000000001e-06 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810041+0000", 2024-09-13T09:19:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0679999999999998e-06 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875256+0000", 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065215765999999994 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875366+0000", 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109639 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875502+0000", 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013630399999999999 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52445 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.535136035999997, 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065733503999999998, 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52445 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810044+0000", 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810043+0000", 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810045+0000", 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.813e-06 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810050+0000", 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5940000000000004e-06 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810054+0000", 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439999999999998e-06 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875543+0000", 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065489393000000007 2024-09-13T09:19:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875658+0000", 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115302 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875777+0000", 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118776 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52446 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8175616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.535122706999999, 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065969204000000004, 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52446 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810057+0000", 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810059+0000", 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.84e-06 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810063+0000", 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4329999999999997e-06 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810067+0000", 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8800000000000001e-06 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875813+0000", 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065745910000000005 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.875903+0000", 2024-09-13T09:19:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0665e-05 2024-09-13T09:19:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876026+0000", 2024-09-13T09:19:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122836 2024-09-13T09:19:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52447 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8273920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:19:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.535088465999998, 2024-09-13T09:19:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066176867, 2024-09-13T09:19:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52447 2024-09-13T09:19:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:19:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:19:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810091+0000", 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810093+0000", 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6089999999999999e-06 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810099+0000", 2024-09-13T09:19:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1539999999999999e-06 2024-09-13T09:19:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810104+0000", 2024-09-13T09:19:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4630000000000003e-06 2024-09-13T09:19:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876062+0000", 2024-09-13T09:19:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.065957672999999994 2024-09-13T09:19:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876153+0000", 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1119000000000006e-05 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876268+0000", 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115431 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52448 3.15 3:ad550f28:::rbd_data.118568665621.000000000000000c:1d [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:19:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.535072298000003, 2024-09-13T09:19:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066534826000000005, 2024-09-13T09:19:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52448 2024-09-13T09:19:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810107+0000", 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810109+0000", 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0169999999999999e-06 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810115+0000", 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:19:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810120+0000", 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9620000000000001e-06 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876349+0000", 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066229213999999995 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876484+0000", 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013503199999999999 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876642+0000", 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158228 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:52452 3.d 3:b5f40919:::rbd_data.118567e41306.0000000000000055:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.53505646, 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070307575999999997, 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 52452 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810123+0000", 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810126+0000", 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810132+0000", 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7349999999999999e-06 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.810136+0000", 2024-09-13T09:19:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2069999999999997e-06 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876306+0000", 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066169943999999994 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.876520+0000", 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021377500000000001 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.877399+0000", 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087836900000000005 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880158+0000", 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002759657 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880212+0000", 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3551999999999998e-05 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880315+0000", 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010306099999999999 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:34.880431+0000", 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011603300000000001 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.447 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:78930 3.13 3:cc588144:::rbd_data.11853dd604c.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-13T09:19:12.447 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:59.956179+0000", 2024-09-13T09:19:12.447 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.389000822, 2024-09-13T09:19:12.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067694713000000004, 2024-09-13T09:19:12.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:12.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:12.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:12.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:12.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78930 2024-09-13T09:19:12.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.447 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956179+0000", 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956179+0000", 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956177+0000", 2024-09-13T09:19:12.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956192+0000", 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5173e-05 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956223+0000", 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1056000000000002e-05 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956242+0000", 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8916999999999999e-05 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956341+0000", 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9011999999999993e-05 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.956470+0000", 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129603 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.969554+0000", 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013083539999999999 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.023628+0000", 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054073884000000003 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.023715+0000", 2024-09-13T09:19:12.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7056000000000001e-05 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.023809+0000", 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4382000000000005e-05 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.023874+0000", 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4403e-05 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:12.453 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:12.453 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:12.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:19:12.455 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:12.453+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:12.556 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:12.553+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:12.588 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:12.588 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:12.588 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:12.588 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:12.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:19:12.700 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:12.697+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:19:12.771 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:12.769+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:12.784 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:12.784 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:12.784 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:12.784 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:12.784 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:12.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:19:12.858 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:12.853+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:12.959 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:12.953+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:12.990 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:12.990 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:12.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:12.990 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:12.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.990 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3:ed699f0f:::rbd_data.118591b29fad.000000000000008c:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:19:12.990 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:12.990 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 90.126290116000007, 2024-09-13T09:19:12.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94878089700000001, 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:12.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:12.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:12.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:12.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:12.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:19:12.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:12.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:12.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:19:12.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:19:12.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:12.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:19:12.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:19:12.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810654+0000", 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94834837599999999 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810755+0000", 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100987 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:12.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810828+0000", 2024-09-13T09:19:12.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2751999999999997e-05 2024-09-13T09:19:12.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:12.996 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:12.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:12.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.996 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19465 3.10 3:0e3a5b30:::rbd_data.1185b72576e4.0000000000000062:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:19:12.996 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:12.996 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.040445379000005, 2024-09-13T09:19:12.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97489159000000003, 2024-09-13T09:19:12.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:12.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19465 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947890+0000", 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947899+0000", 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6840000000000008e-06 2024-09-13T09:19:12.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947917+0000", 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7731e-05 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947925+0000", 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4189999999999996e-06 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:12.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947990+0000", 2024-09-13T09:19:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5909999999999997e-05 2024-09-13T09:19:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947998+0000", 2024-09-13T09:19:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4449999999999999e-06 2024-09-13T09:19:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.919994+0000", 2024-09-13T09:19:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97199595900000002 2024-09-13T09:19:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920258+0000", 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026439500000000001 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920629+0000", 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037096299999999998 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.921797+0000", 2024-09-13T09:19:13.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001167721 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922544+0000", 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00074661200000000001 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922628+0000", 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4016000000000003e-05 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922783+0000", 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015585499999999999 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51771 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243708906000002, 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083783107999999995, 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51771 2024-09-13T09:19:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744631+0000", 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.943e-06 2024-09-13T09:19:13.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744639+0000", 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744646+0000", 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5860000000000001e-06 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828151+0000", 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083505732999999999 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828286+0000", 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013457200000000001 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828411+0000", 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012564999999999999 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.007 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51772 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243685810000002, 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083939185999999999, 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.008 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51772 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744654+0000", 2024-09-13T09:19:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2440000000000002e-06 2024-09-13T09:19:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744662+0000", 2024-09-13T09:19:13.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3659999999999992e-06 2024-09-13T09:19:13.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744669+0000", 2024-09-13T09:19:13.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6529999999999997e-06 2024-09-13T09:19:13.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828455+0000", 2024-09-13T09:19:13.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083785707000000001 2024-09-13T09:19:13.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828512+0000", 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7549000000000001e-05 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828591+0000", 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8274000000000003e-05 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.013 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51773 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.013 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:13.013 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243662153999999, 2024-09-13T09:19:13.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084139096999999996, 2024-09-13T09:19:13.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.013 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.013 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.013 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.013 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.013 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51773 2024-09-13T09:19:13.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.014 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:13.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:13.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744674+0000", 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744677+0000", 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9399999999999998e-06 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744689+0000", 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.153e-05 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744705+0000", 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5679e-05 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828658+0000", 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083953190999999996 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828734+0000", 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5784999999999994e-05 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828814+0000", 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579000000000002e-05 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51774 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243624791999999, 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084301713, 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51774 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:13.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744723+0000", 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0832000000000001e-05 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744733+0000", 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031e-05 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744740+0000", 2024-09-13T09:19:13.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0500000000000003e-06 2024-09-13T09:19:13.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828867+0000", 2024-09-13T09:19:13.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084127417999999995 2024-09-13T09:19:13.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828932+0000", 2024-09-13T09:19:13.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4202000000000006e-05 2024-09-13T09:19:13.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829014+0000", 2024-09-13T09:19:13.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-13T09:19:13.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.022 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.022 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51775 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.022 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243591119999998, 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084465871999999997, 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51775 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:13.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744749+0000", 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5200000000000002e-06 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744757+0000", 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1100000000000003e-06 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744764+0000", 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8430000000000004e-06 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829058+0000", 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084293899000000005 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829111+0000", 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3315e-05 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829212+0000", 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010079200000000001 2024-09-13T09:19:13.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51776 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243567407999997, 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084631102999999999, 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51776 2024-09-13T09:19:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744769+0000", 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744772+0000", 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.14e-06 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744781+0000", 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.276e-06 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744788+0000", 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0269999999999998e-06 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829254+0000", 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084466325999999994 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829310+0000", 2024-09-13T09:19:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5852e-05 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829401+0000", 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1075000000000005e-05 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51777 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.24351214, 2024-09-13T09:19:13.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084901272, 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51777 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744824+0000", 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744828+0000", 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744840+0000", 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1997000000000001e-05 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744847+0000", 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0659999999999998e-06 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829460+0000", 2024-09-13T09:19:13.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084613387999999998 2024-09-13T09:19:13.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829555+0000", 2024-09-13T09:19:13.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4999000000000003e-05 2024-09-13T09:19:13.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829726+0000", 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017100899999999999 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51778 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243459696000002, 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085026334999999995, 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51778 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744877+0000", 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744881+0000", 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5999999999999998e-06 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744892+0000", 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1732999999999999e-05 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744899+0000", 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0759999999999997e-06 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829772+0000", 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084872633000000003 2024-09-13T09:19:13.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829826+0000", 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4330000000000003e-05 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829904+0000", 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7558999999999994e-05 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51779 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243431127000001, 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085200608999999997, 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51779 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.041 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744905+0000", 2024-09-13T09:19:13.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744909+0000", 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3500000000000001e-06 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744917+0000", 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3129999999999999e-06 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744924+0000", 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6059999999999999e-06 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829976+0000", 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085051775999999996 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830030+0000", 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4404000000000003e-05 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830107+0000", 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6796000000000001e-05 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51780 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243407533999999, 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085367775000000007, 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51780 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:13.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744929+0000", 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744932+0000", 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744940+0000", 2024-09-13T09:19:13.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1130000000000001e-06 2024-09-13T09:19:13.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744947+0000", 2024-09-13T09:19:13.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:19:13.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830148+0000", 2024-09-13T09:19:13.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085201099000000002 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830205+0000", 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6623999999999999e-05 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830297+0000", 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2693000000000002e-05 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51781 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243384497999998, 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085544856000000002, 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51781 2024-09-13T09:19:13.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744952+0000", 2024-09-13T09:19:13.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744955+0000", 2024-09-13T09:19:13.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0960000000000001e-06 2024-09-13T09:19:13.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745048+0000", 2024-09-13T09:19:13.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2806999999999994e-05 2024-09-13T09:19:13.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745058+0000", 2024-09-13T09:19:13.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0132999999999999e-05 2024-09-13T09:19:13.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830338+0000", 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085279761999999995 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830420+0000", 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1551000000000006e-05 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830498+0000", 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8103999999999996e-05 2024-09-13T09:19:13.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.055 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.055 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51782 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1794048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.055 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:13.055 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243272576000003, 2024-09-13T09:19:13.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085605889000000004, 2024-09-13T09:19:13.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51782 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:13.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745064+0000", 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745068+0000", 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.523e-06 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745076+0000", 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.354e-06 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745091+0000", 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5274999999999999e-05 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830538+0000", 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085447238999999994 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830591+0000", 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2481000000000001e-05 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830671+0000", 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9716999999999996e-05 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51783 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1892352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243239930000001, 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085732662000000001, 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51783 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745100+0000", 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745108+0000", 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4360000000000002e-06 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745115+0000", 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9269999999999999e-06 2024-09-13T09:19:13.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830707+0000", 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085591133999999999 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830758+0000", 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1180000000000001e-05 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830830+0000", 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2268999999999995e-05 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51784 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.243207962, 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085902834999999997, 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.064 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51784 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745132+0000", 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.36e-06 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745143+0000", 2024-09-13T09:19:13.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745150+0000", 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0400000000000004e-06 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830870+0000", 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085720086000000001 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830955+0000", 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5321000000000005e-05 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831032+0000", 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6811999999999996e-05 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.068 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51785 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.069 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:13.069 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.242653066999999, 2024-09-13T09:19:13.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085542517999999998, 2024-09-13T09:19:13.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.069 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.069 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.069 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.069 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.069 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51785 2024-09-13T09:19:13.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.069 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745160+0000", 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994755 2024-09-13T09:19:13.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745697+0000", 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00053713000000000003 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745731+0000", 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3767999999999999e-05 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745754+0000", 2024-09-13T09:19:13.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3748999999999999e-05 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831075+0000", 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085321067 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831130+0000", 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4156999999999997e-05 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831227+0000", 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7093e-05 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51789 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.239960310999997, 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083633549000000001, 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51789 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:13.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748376+0000", 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748380+0000", 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7529999999999999e-06 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748392+0000", 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2376e-05 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748399+0000", 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0199999999999997e-06 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831866+0000", 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083466636999999996 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831933+0000", 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7144000000000006e-05 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832010+0000", 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7352000000000001e-05 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51790 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.239931745, 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083812485000000006, 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51790 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:13.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:13.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748405+0000", 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748408+0000", 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3859999999999999e-06 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748417+0000", 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:19:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748424+0000", 2024-09-13T09:19:13.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:19:13.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832055+0000", 2024-09-13T09:19:13.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083631469 2024-09-13T09:19:13.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832129+0000", 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4240999999999996e-05 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832218+0000", 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8562999999999999e-05 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51791 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2678784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.239907326000001, 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083959632000000006, 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51791 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748429+0000", 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748432+0000", 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0800000000000002e-06 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748441+0000", 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9120000000000004e-06 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748470+0000", 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569e-05 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832260+0000", 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083790586 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832317+0000", 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6054999999999998e-05 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832390+0000", 2024-09-13T09:19:13.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3033000000000004e-05 2024-09-13T09:19:13.087 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.087 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.087 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.087 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.087 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.087 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:13.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:19:13.089 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:13.081+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:13.190 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:13.185+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:13.207 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:13.207 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:13.207 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:13.207 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:13.221 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:19:13.291 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:13.289+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:13.348 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:13.348 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:13.348 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:13.348 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:13.348 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:13.362 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:19:13.407 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:13.405+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:13.493 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:19:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 67.415985640000002, 2024-09-13T09:19:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:19:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:19:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:19:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:19:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:19:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56102 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5439488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.073144583000001, 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060495140000000003, 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56102 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416845+0000", 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416853+0000", 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416862+0000", 2024-09-13T09:19:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1959999999999997e-06 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416868+0000", 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9719999999999997e-06 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477088+0000", 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060219898000000001 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477154+0000", 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5605000000000006e-05 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477341+0000", 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018687599999999999 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56103 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5537792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.073116519000003, 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060576972999999999, 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56103 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416873+0000", 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416876+0000", 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.993e-06 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416883+0000", 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.477e-06 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416888+0000", 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2580000000000003e-06 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477357+0000", 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060469055000000001 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477396+0000", 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9402e-05 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477451+0000", 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4262000000000003e-05 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56104 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5636096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.073098027, 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060778881999999999, 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56104 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416894+0000", 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4049999999999998e-06 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416900+0000", 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.959e-06 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416905+0000", 2024-09-13T09:19:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-13T09:19:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477465+0000", 2024-09-13T09:19:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060559460000000002 2024-09-13T09:19:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477521+0000", 2024-09-13T09:19:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5813000000000002e-05 2024-09-13T09:19:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477671+0000", 2024-09-13T09:19:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150366 2024-09-13T09:19:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56105 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5734400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.073053815999998, 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060898832, 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56105 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416939+0000", 2024-09-13T09:19:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3550000000000001e-06 2024-09-13T09:19:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416947+0000", 2024-09-13T09:19:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0949999999999996e-06 2024-09-13T09:19:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416953+0000", 2024-09-13T09:19:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6790000000000002e-06 2024-09-13T09:19:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477708+0000", 2024-09-13T09:19:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060755454 2024-09-13T09:19:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477760+0000", 2024-09-13T09:19:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1589999999999999e-05 2024-09-13T09:19:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477835+0000", 2024-09-13T09:19:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5394000000000004e-05 2024-09-13T09:19:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56106 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5832704~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.073032499, 2024-09-13T09:19:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061142999000000003, 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56106 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416957+0000", 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416960+0000", 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6979999999999998e-06 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416967+0000", 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6919999999999996e-06 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416973+0000", 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5679999999999996e-06 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477867+0000", 2024-09-13T09:19:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060894082000000002 2024-09-13T09:19:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477924+0000", 2024-09-13T09:19:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7015999999999998e-05 2024-09-13T09:19:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478101+0000", 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017648300000000001 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56107 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5931008~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.072703885000003, 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060956639999999999, 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56107 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416977+0000", 2024-09-13T09:19:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996905 2024-09-13T09:19:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417291+0000", 2024-09-13T09:19:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031392799999999999 2024-09-13T09:19:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417311+0000", 2024-09-13T09:19:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0343000000000002e-05 2024-09-13T09:19:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417329+0000", 2024-09-13T09:19:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7612e-05 2024-09-13T09:19:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478123+0000", 2024-09-13T09:19:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060794421000000001 2024-09-13T09:19:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478174+0000", 2024-09-13T09:19:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0614e-05 2024-09-13T09:19:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478243+0000", 2024-09-13T09:19:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9226000000000004e-05 2024-09-13T09:19:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56108 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 6029312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.072644400999998, 2024-09-13T09:19:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061163618000000003, 2024-09-13T09:19:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56108 2024-09-13T09:19:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417345+0000", 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417349+0000", 2024-09-13T09:19:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7419999999999999e-06 2024-09-13T09:19:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417367+0000", 2024-09-13T09:19:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7714999999999999e-05 2024-09-13T09:19:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417382+0000", 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5396999999999998e-05 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478270+0000", 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060888022999999999 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478322+0000", 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2234999999999997e-05 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478509+0000", 2024-09-13T09:19:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187109 2024-09-13T09:19:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56121 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7307264~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.068660907999998, 2024-09-13T09:19:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060455939, 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56121 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421332+0000", 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421358+0000", 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6427999999999998e-05 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421374+0000", 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6137000000000002e-05 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481604+0000", 2024-09-13T09:19:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060229528999999997 2024-09-13T09:19:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481680+0000", 2024-09-13T09:19:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6089999999999998e-05 2024-09-13T09:19:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481785+0000", 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105259 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56122 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7405568~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.068587405000002, 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060604366999999999, 2024-09-13T09:19:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56122 2024-09-13T09:19:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421402+0000", 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421406+0000", 2024-09-13T09:19:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.337e-06 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421415+0000", 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421434+0000", 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8617000000000001e-05 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481827+0000", 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060393883000000002 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481910+0000", 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2657999999999994e-05 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482007+0000", 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7057000000000003e-05 2024-09-13T09:19:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56123 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7503872~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.068550866999999, 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060838125999999999, 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56123 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421450+0000", 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1544e-05 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421463+0000", 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421468+0000", 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6250000000000004e-06 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482066+0000", 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060597713999999997 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482163+0000", 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7112000000000001e-05 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482277+0000", 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114274 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56124 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7602176~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.068516912, 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061054929000000001, 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56124 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421476+0000", 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2059999999999999e-06 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421484+0000", 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7889999999999995e-06 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421489+0000", 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482321+0000", 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060831477000000002 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482403+0000", 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2776999999999997e-05 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482528+0000", 2024-09-13T09:19:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012491600000000001 2024-09-13T09:19:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56125 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7700480~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.068496615000001, 2024-09-13T09:19:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061273721000000003, 2024-09-13T09:19:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56125 2024-09-13T09:19:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421493+0000", 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421496+0000", 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.666e-06 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421503+0000", 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421508+0000", 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0919999999999997e-06 2024-09-13T09:19:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482575+0000", 2024-09-13T09:19:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061066339999999997 2024-09-13T09:19:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482659+0000", 2024-09-13T09:19:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4158000000000003e-05 2024-09-13T09:19:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482767+0000", 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010849 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56126 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7798784~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.068477248999997, 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061489349999999998, 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56126 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421531+0000", 2024-09-13T09:19:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8218000000000001e-05 2024-09-13T09:19:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421538+0000", 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.345e-06 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421544+0000", 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482813+0000", 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061268959999999997 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482900+0000", 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6858000000000001e-05 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483002+0000", 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102599 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56127 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7897088~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.068441507999999, 2024-09-13T09:19:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061729777, 2024-09-13T09:19:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56127 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421548+0000", 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421551+0000", 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421559+0000", 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4499999999999998e-06 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421564+0000", 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483067+0000", 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061503073999999998 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483153+0000", 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483278+0000", 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012522499999999999 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56319 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1613824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.062683761999999, 2024-09-13T09:19:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060599663999999998, 2024-09-13T09:19:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56319 2024-09-13T09:19:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427309+0000", 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.672e-06 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427316+0000", 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6850000000000001e-06 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427322+0000", 2024-09-13T09:19:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:19:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487756+0000", 2024-09-13T09:19:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060434817000000002 2024-09-13T09:19:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487825+0000", 2024-09-13T09:19:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8553000000000006e-05 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487906+0000", 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1067000000000002e-05 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56320 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1712128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.062663948000001, 2024-09-13T09:19:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060803930999999999, 2024-09-13T09:19:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56320 2024-09-13T09:19:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427328+0000", 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427379+0000", 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0343999999999998e-05 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427385+0000", 2024-09-13T09:19:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3969999999999999e-06 2024-09-13T09:19:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487941+0000", 2024-09-13T09:19:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060555805999999997 2024-09-13T09:19:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488034+0000", 2024-09-13T09:19:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:19:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488130+0000", 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6348999999999995e-05 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56321 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1810432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.062599011000003, 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060999937999999997, 2024-09-13T09:19:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56321 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427394+0000", 2024-09-13T09:19:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.721e-06 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427403+0000", 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7110000000000001e-06 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427409+0000", 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9710000000000001e-06 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488183+0000", 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060774003 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488291+0000", 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010786899999999999 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488391+0000", 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100142 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56322 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1908736~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.062252895999997, 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060864109999999999, 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56322 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427433+0000", 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996958 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427742+0000", 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000309084 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427754+0000", 2024-09-13T09:19:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1354e-05 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427761+0000", 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8059999999999995e-06 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488429+0000", 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060667139000000002 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488496+0000", 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7595000000000006e-05 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488601+0000", 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105257 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56323 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 2007040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.061647201, 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060546618000000003, 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56323 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427767+0000", 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994245 2024-09-13T09:19:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428348+0000", 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00058032600000000004 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428360+0000", 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2103000000000001e-05 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428367+0000", 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1949999999999997e-06 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488650+0000", 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060282626999999998 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488745+0000", 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5365999999999995e-05 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488890+0000", 2024-09-13T09:19:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000144506 2024-09-13T09:19:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:13.590 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:13.591 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:13.489+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:13.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:19:13.636 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:13.633+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:13.694 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:13.689+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:13.780 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:13.780 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:13.781 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:13.781 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:13.795 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:13.793+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:13.796 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:19:13.897 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:13.889+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:13.969 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:13.969 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:13.969 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:13.969 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:13.969 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:13.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:19:14.027 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:14.025+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:14.098 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:14.093+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:14.106 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:14.106 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:14.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:14.106 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33548 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.113673042999999, 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025760904000000001, 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33548 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990124+0000", 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990130+0000", 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990135+0000", 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007826+0000", 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017691018999999999 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007896+0000", 2024-09-13T09:19:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0059000000000001e-05 2024-09-13T09:19:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.015883+0000", 2024-09-13T09:19:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079875269999999995 2024-09-13T09:19:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33751 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1593344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110253389, 2024-09-13T09:19:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023779101, 2024-09-13T09:19:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33751 2024-09-13T09:19:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993548+0000", 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.776e-06 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993555+0000", 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993560+0000", 2024-09-13T09:19:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:19:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017188+0000", 2024-09-13T09:19:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023627984000000001 2024-09-13T09:19:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017271+0000", 2024-09-13T09:19:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2798000000000002e-05 2024-09-13T09:19:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017321+0000", 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0557999999999999e-05 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33752 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110231269000003, 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023925173000000001, 2024-09-13T09:19:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33752 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993566+0000", 2024-09-13T09:19:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993573+0000", 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.759e-06 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993578+0000", 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8339999999999998e-06 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017349+0000", 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023770959000000001 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017423+0000", 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3652000000000006e-05 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017489+0000", 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6748999999999994e-05 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33753 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1650688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110212787000002, 2024-09-13T09:19:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024066349000000001, 2024-09-13T09:19:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33753 2024-09-13T09:19:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993582+0000", 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993585+0000", 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993591+0000", 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993595+0000", 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017515+0000", 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023920117000000001 2024-09-13T09:19:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017583+0000", 2024-09-13T09:19:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8485999999999994e-05 2024-09-13T09:19:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017649+0000", 2024-09-13T09:19:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:19:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33754 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110196492999997, 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02420106, 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33754 2024-09-13T09:19:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993601+0000", 2024-09-13T09:19:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:19:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993607+0000", 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9669999999999998e-06 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993611+0000", 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017669+0000", 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024057683999999999 2024-09-13T09:19:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017742+0000", 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3397000000000002e-05 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017800+0000", 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7711999999999999e-05 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33755 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1708032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110180073999999, 2024-09-13T09:19:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024346923999999999, 2024-09-13T09:19:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33755 2024-09-13T09:19:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.132 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:14.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:14.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993617+0000", 2024-09-13T09:19:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:19:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993624+0000", 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0469999999999999e-06 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993628+0000", 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017830+0000", 2024-09-13T09:19:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024201879999999999 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017897+0000", 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7096999999999998e-05 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017962+0000", 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5455000000000003e-05 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33756 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110163348, 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024485512000000001, 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33756 2024-09-13T09:19:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993635+0000", 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993641+0000", 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:19:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993645+0000", 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2520000000000001e-06 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017979+0000", 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024333950999999999 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018045+0000", 2024-09-13T09:19:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5843e-05 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018118+0000", 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2417999999999997e-05 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33757 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1765376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110145950000003, 2024-09-13T09:19:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024605578999999999, 2024-09-13T09:19:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33757 2024-09-13T09:19:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993649+0000", 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993651+0000", 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.841e-06 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993657+0000", 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993662+0000", 2024-09-13T09:19:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:19:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018139+0000", 2024-09-13T09:19:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024476748 2024-09-13T09:19:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018196+0000", 2024-09-13T09:19:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7188000000000002e-05 2024-09-13T09:19:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018255+0000", 2024-09-13T09:19:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9045999999999997e-05 2024-09-13T09:19:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33758 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110129317000002, 2024-09-13T09:19:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024736898, 2024-09-13T09:19:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33758 2024-09-13T09:19:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993668+0000", 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993674+0000", 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993679+0000", 2024-09-13T09:19:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.515e-06 2024-09-13T09:19:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018284+0000", 2024-09-13T09:19:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024604816000000002 2024-09-13T09:19:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018356+0000", 2024-09-13T09:19:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2782999999999998e-05 2024-09-13T09:19:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018403+0000", 2024-09-13T09:19:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6726e-05 2024-09-13T09:19:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33759 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1822720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110112710999999, 2024-09-13T09:19:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873635000000002, 2024-09-13T09:19:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33759 2024-09-13T09:19:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993682+0000", 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993685+0000", 2024-09-13T09:19:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:19:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993691+0000", 2024-09-13T09:19:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1840000000000004e-06 2024-09-13T09:19:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993696+0000", 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018429+0000", 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024733330000000001 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018494+0000", 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5233000000000004e-05 2024-09-13T09:19:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018556+0000", 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2125999999999994e-05 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33760 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110095856000001, 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025058980000000002, 2024-09-13T09:19:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33760 2024-09-13T09:19:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993699+0000", 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993702+0000", 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.632e-06 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993708+0000", 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2280000000000003e-06 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993713+0000", 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018586+0000", 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873572 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018645+0000", 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9098000000000002e-05 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018759+0000", 2024-09-13T09:19:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113224 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33761 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1880064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110079159000001, 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025229076, 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33761 2024-09-13T09:19:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993718+0000", 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.092e-06 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993724+0000", 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0839999999999997e-06 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993729+0000", 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018790+0000", 2024-09-13T09:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025061039 2024-09-13T09:19:14.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018871+0000", 2024-09-13T09:19:14.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0885999999999998e-05 2024-09-13T09:19:14.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018945+0000", 2024-09-13T09:19:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4773999999999999e-05 2024-09-13T09:19:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33762 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110062685000003, 2024-09-13T09:19:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025559609, 2024-09-13T09:19:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33762 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993732+0000", 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993735+0000", 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:19:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993740+0000", 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993745+0000", 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5290000000000002e-06 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019043+0000", 2024-09-13T09:19:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025298355000000002 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019134+0000", 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0781999999999999e-05 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019292+0000", 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015829 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33763 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110046744000002, 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025732657999999999, 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33763 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993748+0000", 2024-09-13T09:19:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993755+0000", 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993762+0000", 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993767+0000", 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7230000000000004e-06 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019322+0000", 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025555564999999999 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019402+0000", 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9433999999999999e-05 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019481+0000", 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9739000000000003e-05 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33764 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110024369999998, 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025908526000000001, 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33764 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993773+0000", 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993779+0000", 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9610000000000002e-06 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993783+0000", 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.459e-06 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019513+0000", 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025729399 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019609+0000", 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6504999999999998e-05 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019680+0000", 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0393000000000002e-05 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33765 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1994752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.110008135000001, 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026051009999999999, 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33765 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:14.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:14.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993789+0000", 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993795+0000", 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.088e-06 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993799+0000", 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019704+0000", 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025904311999999999 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019766+0000", 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2829000000000004e-05 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019838+0000", 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1935999999999997e-05 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33766 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.109992079999998, 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026238971999999999, 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.184 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.184 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.184 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.184 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33766 2024-09-13T09:19:14.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.184 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:14.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993805+0000", 2024-09-13T09:19:14.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993811+0000", 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9000000000000003e-06 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993816+0000", 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.527e-06 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019877+0000", 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026061337 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019960+0000", 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3245999999999995e-05 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020042+0000", 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2053999999999997e-05 2024-09-13T09:19:14.187 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.188 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.188 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.188 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33767 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.188 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:14.188 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.109975669999997, 2024-09-13T09:19:14.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026407013999999999, 2024-09-13T09:19:14.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.188 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.188 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.188 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33767 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993819+0000", 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993822+0000", 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993828+0000", 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993832+0000", 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.493e-06 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020072+0000", 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026239519999999999 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020151+0000", 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9278999999999998e-05 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020227+0000", 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5965999999999997e-05 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33768 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:14.192 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.109957217000002, 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026564424, 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33768 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:14.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993840+0000", 2024-09-13T09:19:14.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.996e-06 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993846+0000", 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993858+0000", 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2214e-05 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020253+0000", 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026394926999999999 2024-09-13T09:19:14.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020329+0000", 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5193999999999999e-05 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020403+0000", 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4145000000000002e-05 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:78829 3.6 3:6418b130:::rbd_data.11853dd604c.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.887271023, 2024-09-13T09:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041141776999999997, 2024-09-13T09:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 78829 2024-09-13T09:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216522+0000", 2024-09-13T09:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216540+0000", 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8280000000000001e-05 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216577+0000", 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7151999999999999e-05 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216594+0000", 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7113e-05 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216696+0000", 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010190400000000001 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216833+0000", 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136697 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.224657+0000", 2024-09-13T09:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078241000000000005 2024-09-13T09:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257365+0000", 2024-09-13T09:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032707984000000002 2024-09-13T09:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257404+0000", 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8979000000000001e-05 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257549+0000", 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000145737 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257666+0000", 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116754 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.203 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:14.203 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:14.203 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:14.203 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:19:14.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:19:14.205 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:14.201+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:14.306 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:14.301+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:14.373 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:14.373 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:14.373 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:14.373 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:14.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:19:14.404 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:14.404 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:14.406 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:14.401+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:14.421 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:14.421 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:14.437 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:14.437 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:14.453 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:14.474 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:14.474 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:14.494 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:14.507 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:14.507 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:14.507 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:14.501+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:14.515 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:14.515 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:14.521 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:19:14.535 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:14.608 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:14.601+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:14.666 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:14.667 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:14.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:19:14.709 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:14.705+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:14.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:19:14.810 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:14.805+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:14.830 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:14.830 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:14.847 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:19:14.911 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:14.909+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:15.012 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:15.009+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:15.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:19:15.113 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:15.109+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:15.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:19:15.230 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:15.225+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:15.308 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:15.308 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:15.315 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:15.309+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:15.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:19:15.416 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:15.409+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:15.493 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:15.493 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:15.507 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:19:15.516 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:15.513+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:15.617 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:15.609+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:15.674 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:15.674 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:15.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:19:15.730 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:15.729+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:19:15.762 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:19:15.762 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:19:15.762 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-13T09:19:15.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-09-13T09:19:15.835 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:15.833+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:15.871 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:15.871 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:15.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:19:15.919 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:15.917+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:16.015 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:16.015 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:16.020 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:16.013+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:16.030 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:19:16.088 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-09-13T09:19:16.085+0000 7f1d86e9f640 -1 mon.a@0(leader).osd e113 definitely_dead 0 2024-09-13T09:19:16.122 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:16.117+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:16.202 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:16.202 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:16.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:19:16.258 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:16.257+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:16.322 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:16.317+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:16.386 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:16.387 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:16.401 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:19:16.446 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:16.441+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:16.524 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:16.521+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:16.528 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:16.528 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:16.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:19:16.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:19:16.630 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:16.625+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:16.711 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:16.711 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:16.712 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:16.712 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:16.726 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:19:16.727 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:16.721+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:16.729 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:16.730 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:16.746 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:16.746 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:marked in osd.6. 2024-09-13T09:19:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:16.774 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:16.791 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-13T09:19:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:16.799 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:16.819 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:16.819 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:16.827 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:16.821+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:16.840 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:16.840 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:16.862 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:16.863 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:16.863 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:16.863 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:16.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:19:16.928 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:16.925+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:17.029 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:17.025+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:17.043 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:17.043 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:17.043 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:17.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:19:17.130 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:17.125+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:17.185 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3:ed699f0f:::rbd_data.118591b29fad.000000000000008c:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:19:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.321073037000005, 2024-09-13T09:19:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94878089700000001, 2024-09-13T09:19:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:19:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:19:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:19:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:19:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:19:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810654+0000", 2024-09-13T09:19:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94834837599999999 2024-09-13T09:19:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810755+0000", 2024-09-13T09:19:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100987 2024-09-13T09:19:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810828+0000", 2024-09-13T09:19:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2751999999999997e-05 2024-09-13T09:19:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19465 3.10 3:0e3a5b30:::rbd_data.1185b72576e4.0000000000000062:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.235228300000003, 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97489159000000003, 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19465 2024-09-13T09:19:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947890+0000", 2024-09-13T09:19:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947899+0000", 2024-09-13T09:19:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6840000000000008e-06 2024-09-13T09:19:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947917+0000", 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7731e-05 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947925+0000", 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4189999999999996e-06 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947990+0000", 2024-09-13T09:19:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5909999999999997e-05 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947998+0000", 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4449999999999999e-06 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.919994+0000", 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97199595900000002 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920258+0000", 2024-09-13T09:19:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026439500000000001 2024-09-13T09:19:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920629+0000", 2024-09-13T09:19:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037096299999999998 2024-09-13T09:19:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.921797+0000", 2024-09-13T09:19:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001167721 2024-09-13T09:19:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922544+0000", 2024-09-13T09:19:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00074661200000000001 2024-09-13T09:19:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922628+0000", 2024-09-13T09:19:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4016000000000003e-05 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922783+0000", 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015585499999999999 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51771 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438491827, 2024-09-13T09:19:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083783107999999995, 2024-09-13T09:19:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51771 2024-09-13T09:19:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744631+0000", 2024-09-13T09:19:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.943e-06 2024-09-13T09:19:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744639+0000", 2024-09-13T09:19:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:19:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744646+0000", 2024-09-13T09:19:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5860000000000001e-06 2024-09-13T09:19:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828151+0000", 2024-09-13T09:19:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083505732999999999 2024-09-13T09:19:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828286+0000", 2024-09-13T09:19:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013457200000000001 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828411+0000", 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012564999999999999 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51772 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438468731, 2024-09-13T09:19:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083939185999999999, 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51772 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744654+0000", 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2440000000000002e-06 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744662+0000", 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3659999999999992e-06 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744669+0000", 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6529999999999997e-06 2024-09-13T09:19:17.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828455+0000", 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083785707000000001 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828512+0000", 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7549000000000001e-05 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828591+0000", 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8274000000000003e-05 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51773 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438445074999997, 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084139096999999996, 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.208 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.209 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.209 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.209 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51773 2024-09-13T09:19:17.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.209 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:17.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744674+0000", 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744677+0000", 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9399999999999998e-06 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744689+0000", 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.153e-05 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744705+0000", 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5679e-05 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828658+0000", 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083953190999999996 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828734+0000", 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5784999999999994e-05 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828814+0000", 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579000000000002e-05 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.212 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51774 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438407712999997, 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084301713, 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51774 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744723+0000", 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0832000000000001e-05 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744733+0000", 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031e-05 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744740+0000", 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0500000000000003e-06 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828867+0000", 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084127417999999995 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828932+0000", 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4202000000000006e-05 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829014+0000", 2024-09-13T09:19:17.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-13T09:19:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51775 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438374041000003, 2024-09-13T09:19:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084465871999999997, 2024-09-13T09:19:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51775 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744749+0000", 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5200000000000002e-06 2024-09-13T09:19:17.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744757+0000", 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1100000000000003e-06 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744764+0000", 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8430000000000004e-06 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829058+0000", 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084293899000000005 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829111+0000", 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3315e-05 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829212+0000", 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010079200000000001 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.221 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51776 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438350329000002, 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084631102999999999, 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51776 2024-09-13T09:19:17.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744769+0000", 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744772+0000", 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.14e-06 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744781+0000", 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.276e-06 2024-09-13T09:19:17.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744788+0000", 2024-09-13T09:19:17.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0269999999999998e-06 2024-09-13T09:19:17.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829254+0000", 2024-09-13T09:19:17.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084466325999999994 2024-09-13T09:19:17.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829310+0000", 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5852e-05 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829401+0000", 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1075000000000005e-05 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.226 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51777 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438295060999998, 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084901272, 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51777 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744824+0000", 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:17.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744828+0000", 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744840+0000", 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1997000000000001e-05 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744847+0000", 2024-09-13T09:19:17.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0659999999999998e-06 2024-09-13T09:19:17.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829460+0000", 2024-09-13T09:19:17.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084613387999999998 2024-09-13T09:19:17.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829555+0000", 2024-09-13T09:19:17.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4999000000000003e-05 2024-09-13T09:19:17.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829726+0000", 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017100899999999999 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51778 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438242617, 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085026334999999995, 2024-09-13T09:19:17.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51778 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744877+0000", 2024-09-13T09:19:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744881+0000", 2024-09-13T09:19:17.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5999999999999998e-06 2024-09-13T09:19:17.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744892+0000", 2024-09-13T09:19:17.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1732999999999999e-05 2024-09-13T09:19:17.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744899+0000", 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0759999999999997e-06 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829772+0000", 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084872633000000003 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829826+0000", 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4330000000000003e-05 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829904+0000", 2024-09-13T09:19:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7558999999999994e-05 2024-09-13T09:19:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51779 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438214047999999, 2024-09-13T09:19:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085200608999999997, 2024-09-13T09:19:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51779 2024-09-13T09:19:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744905+0000", 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744909+0000", 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3500000000000001e-06 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744917+0000", 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3129999999999999e-06 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744924+0000", 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6059999999999999e-06 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829976+0000", 2024-09-13T09:19:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085051775999999996 2024-09-13T09:19:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830030+0000", 2024-09-13T09:19:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4404000000000003e-05 2024-09-13T09:19:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830107+0000", 2024-09-13T09:19:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6796000000000001e-05 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51780 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438190454999997, 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085367775000000007, 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51780 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744929+0000", 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744932+0000", 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744940+0000", 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1130000000000001e-06 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744947+0000", 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830148+0000", 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085201099000000002 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830205+0000", 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6623999999999999e-05 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830297+0000", 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2693000000000002e-05 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51781 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438167419000003, 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085544856000000002, 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51781 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744952+0000", 2024-09-13T09:19:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744955+0000", 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0960000000000001e-06 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745048+0000", 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2806999999999994e-05 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745058+0000", 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0132999999999999e-05 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830338+0000", 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085279761999999995 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830420+0000", 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1551000000000006e-05 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830498+0000", 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8103999999999996e-05 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51782 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1794048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438055497000001, 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085605889000000004, 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51782 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745064+0000", 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745068+0000", 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.523e-06 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745076+0000", 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.354e-06 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745091+0000", 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5274999999999999e-05 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830538+0000", 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085447238999999994 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830591+0000", 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2481000000000001e-05 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830671+0000", 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9716999999999996e-05 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.254 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51783 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1892352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.438022850999999, 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085732662000000001, 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51783 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745100+0000", 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745108+0000", 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4360000000000002e-06 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745115+0000", 2024-09-13T09:19:17.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9269999999999999e-06 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830707+0000", 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085591133999999999 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830758+0000", 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1180000000000001e-05 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830830+0000", 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2268999999999995e-05 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51784 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.437990882999998, 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085902834999999997, 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51784 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745132+0000", 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.36e-06 2024-09-13T09:19:17.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745143+0000", 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745150+0000", 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0400000000000004e-06 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830870+0000", 2024-09-13T09:19:17.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085720086000000001 2024-09-13T09:19:17.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830955+0000", 2024-09-13T09:19:17.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5321000000000005e-05 2024-09-13T09:19:17.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831032+0000", 2024-09-13T09:19:17.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6811999999999996e-05 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51785 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.437435987999997, 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085542517999999998, 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.264 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51785 2024-09-13T09:19:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745160+0000", 2024-09-13T09:19:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994755 2024-09-13T09:19:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745697+0000", 2024-09-13T09:19:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00053713000000000003 2024-09-13T09:19:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745731+0000", 2024-09-13T09:19:17.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3767999999999999e-05 2024-09-13T09:19:17.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745754+0000", 2024-09-13T09:19:17.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3748999999999999e-05 2024-09-13T09:19:17.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831075+0000", 2024-09-13T09:19:17.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085321067 2024-09-13T09:19:17.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831130+0000", 2024-09-13T09:19:17.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4156999999999997e-05 2024-09-13T09:19:17.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831227+0000", 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7093e-05 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51789 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.434743232000002, 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083633549000000001, 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51789 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748376+0000", 2024-09-13T09:19:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748380+0000", 2024-09-13T09:19:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7529999999999999e-06 2024-09-13T09:19:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748392+0000", 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2376e-05 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748399+0000", 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0199999999999997e-06 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831866+0000", 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083466636999999996 2024-09-13T09:19:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831933+0000", 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7144000000000006e-05 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832010+0000", 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7352000000000001e-05 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51790 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.434714665999998, 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083812485000000006, 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51790 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748405+0000", 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748408+0000", 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3859999999999999e-06 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748417+0000", 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748424+0000", 2024-09-13T09:19:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:19:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832055+0000", 2024-09-13T09:19:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083631469 2024-09-13T09:19:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832129+0000", 2024-09-13T09:19:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4240999999999996e-05 2024-09-13T09:19:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832218+0000", 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8562999999999999e-05 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51791 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2678784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.434690246999999, 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083959632000000006, 2024-09-13T09:19:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51791 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748429+0000", 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748432+0000", 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0800000000000002e-06 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748441+0000", 2024-09-13T09:19:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9120000000000004e-06 2024-09-13T09:19:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748470+0000", 2024-09-13T09:19:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569e-05 2024-09-13T09:19:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832260+0000", 2024-09-13T09:19:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083790586 2024-09-13T09:19:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832317+0000", 2024-09-13T09:19:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6054999999999998e-05 2024-09-13T09:19:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832390+0000", 2024-09-13T09:19:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3033000000000004e-05 2024-09-13T09:19:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.285 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:17.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:19:17.288 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:17.285+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:17.387 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:17.385+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:17.464 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:17.464 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:17.465 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:17.465 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:17.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:19:17.524 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:17.520+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:17.589 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:17.584+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:17.598 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:17.599 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:17.599 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:17.599 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:17.599 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:17.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:19:17.700 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:17.696+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:17.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:19:17.791 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:17.789+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:17.794 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.718106887000005, 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:19:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:19:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:19:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:19:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:19:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:19:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:19:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:19:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:19:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56102 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5439488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.375265830000004, 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060495140000000003, 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56102 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416845+0000", 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416853+0000", 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416862+0000", 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1959999999999997e-06 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416868+0000", 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9719999999999997e-06 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477088+0000", 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060219898000000001 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477154+0000", 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5605000000000006e-05 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477341+0000", 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018687599999999999 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56103 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5537792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.375237765999998, 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060576972999999999, 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56103 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416873+0000", 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416876+0000", 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.993e-06 2024-09-13T09:19:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416883+0000", 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.477e-06 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416888+0000", 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2580000000000003e-06 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477357+0000", 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060469055000000001 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477396+0000", 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9402e-05 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477451+0000", 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4262000000000003e-05 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56104 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5636096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.375219274000003, 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060778881999999999, 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56104 2024-09-13T09:19:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416894+0000", 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4049999999999998e-06 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416900+0000", 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.959e-06 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416905+0000", 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477465+0000", 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060559460000000002 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477521+0000", 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5813000000000002e-05 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477671+0000", 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150366 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56105 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5734400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.375175063, 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060898832, 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56105 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416939+0000", 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3550000000000001e-06 2024-09-13T09:19:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416947+0000", 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0949999999999996e-06 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416953+0000", 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6790000000000002e-06 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477708+0000", 2024-09-13T09:19:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060755454 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477760+0000", 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1589999999999999e-05 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477835+0000", 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5394000000000004e-05 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56106 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5832704~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.375153746000002, 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061142999000000003, 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56106 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416957+0000", 2024-09-13T09:19:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416960+0000", 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6979999999999998e-06 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416967+0000", 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6919999999999996e-06 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416973+0000", 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5679999999999996e-06 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477867+0000", 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060894082000000002 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477924+0000", 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7015999999999998e-05 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478101+0000", 2024-09-13T09:19:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017648300000000001 2024-09-13T09:19:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56107 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5931008~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.374825131999998, 2024-09-13T09:19:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060956639999999999, 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56107 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416977+0000", 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996905 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417291+0000", 2024-09-13T09:19:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031392799999999999 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417311+0000", 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0343000000000002e-05 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417329+0000", 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7612e-05 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478123+0000", 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060794421000000001 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478174+0000", 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0614e-05 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478243+0000", 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9226000000000004e-05 2024-09-13T09:19:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56108 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 6029312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.374765648, 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061163618000000003, 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56108 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417345+0000", 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417349+0000", 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7419999999999999e-06 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417367+0000", 2024-09-13T09:19:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7714999999999999e-05 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417382+0000", 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5396999999999998e-05 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478270+0000", 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060888022999999999 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478322+0000", 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2234999999999997e-05 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478509+0000", 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187109 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56121 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7307264~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.370782155000001, 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060455939, 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56121 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421332+0000", 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421358+0000", 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6427999999999998e-05 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421374+0000", 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6137000000000002e-05 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481604+0000", 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060229528999999997 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481680+0000", 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6089999999999998e-05 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481785+0000", 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105259 2024-09-13T09:19:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56122 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7405568~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.370708651999998, 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060604366999999999, 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56122 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421402+0000", 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421406+0000", 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.337e-06 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421415+0000", 2024-09-13T09:19:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:19:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421434+0000", 2024-09-13T09:19:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8617000000000001e-05 2024-09-13T09:19:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481827+0000", 2024-09-13T09:19:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060393883000000002 2024-09-13T09:19:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481910+0000", 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2657999999999994e-05 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482007+0000", 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7057000000000003e-05 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56123 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7503872~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.370672114000001, 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060838125999999999, 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56123 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421450+0000", 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1544e-05 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421463+0000", 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421468+0000", 2024-09-13T09:19:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6250000000000004e-06 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482066+0000", 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060597713999999997 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482163+0000", 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7112000000000001e-05 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482277+0000", 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114274 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56124 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7602176~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.370638159000002, 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061054929000000001, 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56124 2024-09-13T09:19:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421476+0000", 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2059999999999999e-06 2024-09-13T09:19:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421484+0000", 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7889999999999995e-06 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421489+0000", 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482321+0000", 2024-09-13T09:19:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060831477000000002 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482403+0000", 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2776999999999997e-05 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482528+0000", 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012491600000000001 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56125 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7700480~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.370617862000003, 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061273721000000003, 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56125 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421493+0000", 2024-09-13T09:19:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421496+0000", 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.666e-06 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421503+0000", 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421508+0000", 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0919999999999997e-06 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482575+0000", 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061066339999999997 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482659+0000", 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4158000000000003e-05 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482767+0000", 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010849 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56126 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7798784~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.370598495999999, 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061489349999999998, 2024-09-13T09:19:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56126 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421531+0000", 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8218000000000001e-05 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421538+0000", 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.345e-06 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421544+0000", 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482813+0000", 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061268959999999997 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482900+0000", 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6858000000000001e-05 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483002+0000", 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102599 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56127 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7897088~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.370562755000002, 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061729777, 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56127 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421548+0000", 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421551+0000", 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421559+0000", 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4499999999999998e-06 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421564+0000", 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483067+0000", 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061503073999999998 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483153+0000", 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483278+0000", 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012522499999999999 2024-09-13T09:19:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56319 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1613824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.364805009000001, 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060599663999999998, 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.865 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56319 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427309+0000", 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.672e-06 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427316+0000", 2024-09-13T09:19:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6850000000000001e-06 2024-09-13T09:19:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427322+0000", 2024-09-13T09:19:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:19:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487756+0000", 2024-09-13T09:19:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060434817000000002 2024-09-13T09:19:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487825+0000", 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8553000000000006e-05 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487906+0000", 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1067000000000002e-05 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56320 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1712128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.364785195000003, 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060803930999999999, 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56320 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427328+0000", 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427379+0000", 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0343999999999998e-05 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427385+0000", 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3969999999999999e-06 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487941+0000", 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060555805999999997 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488034+0000", 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488130+0000", 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6348999999999995e-05 2024-09-13T09:19:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56321 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1810432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.364720257999998, 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060999937999999997, 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.874 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56321 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427394+0000", 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.721e-06 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427403+0000", 2024-09-13T09:19:17.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7110000000000001e-06 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427409+0000", 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9710000000000001e-06 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488183+0000", 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060774003 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488291+0000", 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010786899999999999 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488391+0000", 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100142 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.878 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56322 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1908736~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.364374142999999, 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060864109999999999, 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56322 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427433+0000", 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996958 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427742+0000", 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000309084 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427754+0000", 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1354e-05 2024-09-13T09:19:17.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427761+0000", 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8059999999999995e-06 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488429+0000", 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060667139000000002 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488496+0000", 2024-09-13T09:19:17.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7595000000000006e-05 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488601+0000", 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105257 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56323 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 2007040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.363768448000002, 2024-09-13T09:19:17.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060546618000000003, 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56323 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427767+0000", 2024-09-13T09:19:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994245 2024-09-13T09:19:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428348+0000", 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00058032600000000004 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428360+0000", 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2103000000000001e-05 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428367+0000", 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1949999999999997e-06 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488650+0000", 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060282626999999998 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488745+0000", 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5365999999999995e-05 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488890+0000", 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000144506 2024-09-13T09:19:17.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.888 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:17.888 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:17.888 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:17.889 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:19:17.893 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:17.885+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:17.993 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:17.989+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:18.049 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:18.049 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:18.049 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:18.049 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:18.064 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:19:18.107 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:18.104+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:18.182 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:18.183 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:18.195 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:18.193+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:18.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:19:18.297 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:18.293+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:18.371 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:18.371 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:18.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:18.371 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:18.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.371 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33548 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.371 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:18.372 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.377951037999999, 2024-09-13T09:19:18.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025760904000000001, 2024-09-13T09:19:18.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.372 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.372 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.372 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.372 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33548 2024-09-13T09:19:18.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.372 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:18.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:18.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990124+0000", 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990130+0000", 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990135+0000", 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007826+0000", 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017691018999999999 2024-09-13T09:19:18.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007896+0000", 2024-09-13T09:19:18.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0059000000000001e-05 2024-09-13T09:19:18.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.015883+0000", 2024-09-13T09:19:18.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079875269999999995 2024-09-13T09:19:18.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.376 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.377 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33751 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1593344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.377 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:18.377 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374531384000001, 2024-09-13T09:19:18.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023779101, 2024-09-13T09:19:18.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.377 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33751 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993548+0000", 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.776e-06 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993555+0000", 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:19:18.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993560+0000", 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017188+0000", 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023627984000000001 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017271+0000", 2024-09-13T09:19:18.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2798000000000002e-05 2024-09-13T09:19:18.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017321+0000", 2024-09-13T09:19:18.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0557999999999999e-05 2024-09-13T09:19:18.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.381 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33752 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374509263999997, 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023925173000000001, 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33752 2024-09-13T09:19:18.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.383 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:18.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:18.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993566+0000", 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993573+0000", 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.759e-06 2024-09-13T09:19:18.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993578+0000", 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8339999999999998e-06 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017349+0000", 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023770959000000001 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017423+0000", 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3652000000000006e-05 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017489+0000", 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6748999999999994e-05 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33753 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1650688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.386 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:18.387 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374490782000002, 2024-09-13T09:19:18.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024066349000000001, 2024-09-13T09:19:18.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.387 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.387 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.387 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.387 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.387 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33753 2024-09-13T09:19:18.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.388 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:18.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993582+0000", 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993585+0000", 2024-09-13T09:19:18.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:19:18.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993591+0000", 2024-09-13T09:19:18.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:19:18.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993595+0000", 2024-09-13T09:19:18.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:19:18.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017515+0000", 2024-09-13T09:19:18.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023920117000000001 2024-09-13T09:19:18.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017583+0000", 2024-09-13T09:19:18.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8485999999999994e-05 2024-09-13T09:19:18.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017649+0000", 2024-09-13T09:19:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:19:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33754 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374474487999997, 2024-09-13T09:19:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02420106, 2024-09-13T09:19:18.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.393 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.393 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33754 2024-09-13T09:19:18.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.393 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:18.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:18.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993601+0000", 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993607+0000", 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9669999999999998e-06 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993611+0000", 2024-09-13T09:19:18.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:19:18.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017669+0000", 2024-09-13T09:19:18.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024057683999999999 2024-09-13T09:19:18.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017742+0000", 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3397000000000002e-05 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017800+0000", 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7711999999999999e-05 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33755 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1708032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.397 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:18.398 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374458068999999, 2024-09-13T09:19:18.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024346923999999999, 2024-09-13T09:19:18.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.398 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33755 2024-09-13T09:19:18.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.398 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993617+0000", 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993624+0000", 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0469999999999999e-06 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993628+0000", 2024-09-13T09:19:18.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:19:18.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017830+0000", 2024-09-13T09:19:18.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024201879999999999 2024-09-13T09:19:18.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017897+0000", 2024-09-13T09:19:18.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7096999999999998e-05 2024-09-13T09:19:18.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017962+0000", 2024-09-13T09:19:18.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5455000000000003e-05 2024-09-13T09:19:18.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.402 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.403 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33756 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.403 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:18.403 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374441343000001, 2024-09-13T09:19:18.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024485512000000001, 2024-09-13T09:19:18.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.403 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33756 2024-09-13T09:19:18.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.404 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:18.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:18.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:18.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993635+0000", 2024-09-13T09:19:18.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:19:18.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993641+0000", 2024-09-13T09:19:18.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:19:18.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993645+0000", 2024-09-13T09:19:18.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2520000000000001e-06 2024-09-13T09:19:18.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017979+0000", 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024333950999999999 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018045+0000", 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5843e-05 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018118+0000", 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2417999999999997e-05 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.408 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33757 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1765376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.408 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:18.408 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374423944999997, 2024-09-13T09:19:18.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024605578999999999, 2024-09-13T09:19:18.408 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.408 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33757 2024-09-13T09:19:18.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.409 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:18.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:18.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993649+0000", 2024-09-13T09:19:18.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993651+0000", 2024-09-13T09:19:18.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.841e-06 2024-09-13T09:19:18.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993657+0000", 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993662+0000", 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018139+0000", 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024476748 2024-09-13T09:19:18.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018196+0000", 2024-09-13T09:19:18.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7188000000000002e-05 2024-09-13T09:19:18.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018255+0000", 2024-09-13T09:19:18.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9045999999999997e-05 2024-09-13T09:19:18.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.413 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.413 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33758 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.413 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:18.413 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374407312000002, 2024-09-13T09:19:18.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024736898, 2024-09-13T09:19:18.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.414 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.414 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33758 2024-09-13T09:19:18.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.414 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:18.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:18.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:18.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993668+0000", 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993674+0000", 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993679+0000", 2024-09-13T09:19:18.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.515e-06 2024-09-13T09:19:18.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018284+0000", 2024-09-13T09:19:18.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024604816000000002 2024-09-13T09:19:18.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018356+0000", 2024-09-13T09:19:18.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2782999999999998e-05 2024-09-13T09:19:18.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018403+0000", 2024-09-13T09:19:18.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6726e-05 2024-09-13T09:19:18.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.418 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.418 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33759 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1822720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.418 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:18.419 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374390706, 2024-09-13T09:19:18.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873635000000002, 2024-09-13T09:19:18.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.419 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33759 2024-09-13T09:19:18.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.419 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993682+0000", 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993685+0000", 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993691+0000", 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1840000000000004e-06 2024-09-13T09:19:18.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993696+0000", 2024-09-13T09:19:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:19:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018429+0000", 2024-09-13T09:19:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024733330000000001 2024-09-13T09:19:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018494+0000", 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5233000000000004e-05 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018556+0000", 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2125999999999994e-05 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.423 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33760 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374373851000001, 2024-09-13T09:19:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025058980000000002, 2024-09-13T09:19:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33760 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993699+0000", 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993702+0000", 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.632e-06 2024-09-13T09:19:18.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993708+0000", 2024-09-13T09:19:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2280000000000003e-06 2024-09-13T09:19:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993713+0000", 2024-09-13T09:19:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:19:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018586+0000", 2024-09-13T09:19:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873572 2024-09-13T09:19:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018645+0000", 2024-09-13T09:19:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9098000000000002e-05 2024-09-13T09:19:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018759+0000", 2024-09-13T09:19:18.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113224 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33761 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1880064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374357154000002, 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025229076, 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33761 2024-09-13T09:19:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993718+0000", 2024-09-13T09:19:18.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.092e-06 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993724+0000", 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0839999999999997e-06 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993729+0000", 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018790+0000", 2024-09-13T09:19:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025061039 2024-09-13T09:19:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018871+0000", 2024-09-13T09:19:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0885999999999998e-05 2024-09-13T09:19:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018945+0000", 2024-09-13T09:19:18.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4773999999999999e-05 2024-09-13T09:19:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33762 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374340680000003, 2024-09-13T09:19:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025559609, 2024-09-13T09:19:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33762 2024-09-13T09:19:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:18.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993732+0000", 2024-09-13T09:19:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993735+0000", 2024-09-13T09:19:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:19:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993740+0000", 2024-09-13T09:19:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993745+0000", 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5290000000000002e-06 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019043+0000", 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025298355000000002 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019134+0000", 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0781999999999999e-05 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019292+0000", 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015829 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33763 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.440 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374324739000002, 2024-09-13T09:19:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025732657999999999, 2024-09-13T09:19:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33763 2024-09-13T09:19:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.442 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993748+0000", 2024-09-13T09:19:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993755+0000", 2024-09-13T09:19:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:19:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993762+0000", 2024-09-13T09:19:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:19:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993767+0000", 2024-09-13T09:19:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7230000000000004e-06 2024-09-13T09:19:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019322+0000", 2024-09-13T09:19:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025555564999999999 2024-09-13T09:19:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019402+0000", 2024-09-13T09:19:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9433999999999999e-05 2024-09-13T09:19:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019481+0000", 2024-09-13T09:19:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9739000000000003e-05 2024-09-13T09:19:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33764 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374302364999998, 2024-09-13T09:19:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025908526000000001, 2024-09-13T09:19:18.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33764 2024-09-13T09:19:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993773+0000", 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993779+0000", 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9610000000000002e-06 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993783+0000", 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.459e-06 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019513+0000", 2024-09-13T09:19:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025729399 2024-09-13T09:19:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019609+0000", 2024-09-13T09:19:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6504999999999998e-05 2024-09-13T09:19:18.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019680+0000", 2024-09-13T09:19:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0393000000000002e-05 2024-09-13T09:19:18.453 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33765 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1994752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.454 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374286130000002, 2024-09-13T09:19:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026051009999999999, 2024-09-13T09:19:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.455 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33765 2024-09-13T09:19:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:18.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993789+0000", 2024-09-13T09:19:18.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:19:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993795+0000", 2024-09-13T09:19:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.088e-06 2024-09-13T09:19:18.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993799+0000", 2024-09-13T09:19:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:19:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019704+0000", 2024-09-13T09:19:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025904311999999999 2024-09-13T09:19:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019766+0000", 2024-09-13T09:19:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2829000000000004e-05 2024-09-13T09:19:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019838+0000", 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1935999999999997e-05 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33766 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374270074999998, 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026238971999999999, 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33766 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993805+0000", 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993811+0000", 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9000000000000003e-06 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993816+0000", 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.527e-06 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019877+0000", 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026061337 2024-09-13T09:19:18.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019960+0000", 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3245999999999995e-05 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020042+0000", 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2053999999999997e-05 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.466 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33767 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374253664999998, 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026407013999999999, 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33767 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:18.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993819+0000", 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993822+0000", 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993828+0000", 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993832+0000", 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.493e-06 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020072+0000", 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026239519999999999 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020151+0000", 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9278999999999998e-05 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020227+0000", 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5965999999999997e-05 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.470 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33768 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.374235212000002, 2024-09-13T09:19:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026564424, 2024-09-13T09:19:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.471 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33768 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:18.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993840+0000", 2024-09-13T09:19:18.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.996e-06 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993846+0000", 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993858+0000", 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2214e-05 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020253+0000", 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026394926999999999 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020329+0000", 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5193999999999999e-05 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020403+0000", 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4145000000000002e-05 2024-09-13T09:19:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:78829 3.6 3:6418b130:::rbd_data.11853dd604c.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.151549018000001, 2024-09-13T09:19:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041141776999999997, 2024-09-13T09:19:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 78829 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216522+0000", 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216540+0000", 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8280000000000001e-05 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216577+0000", 2024-09-13T09:19:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7151999999999999e-05 2024-09-13T09:19:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216594+0000", 2024-09-13T09:19:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7113e-05 2024-09-13T09:19:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216696+0000", 2024-09-13T09:19:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010190400000000001 2024-09-13T09:19:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216833+0000", 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136697 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.224657+0000", 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078241000000000005 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257365+0000", 2024-09-13T09:19:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032707984000000002 2024-09-13T09:19:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257404+0000", 2024-09-13T09:19:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8979000000000001e-05 2024-09-13T09:19:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257549+0000", 2024-09-13T09:19:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000145737 2024-09-13T09:19:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257666+0000", 2024-09-13T09:19:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116754 2024-09-13T09:19:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.483 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:18.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:19:18.485 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:18.480+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:18.585 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:18.580+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:18.660 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:18.661 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:18.661 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:18.661 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:18.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:19:18.716 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:18.712+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-13T09:19:18.786 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:18.780+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:18.847 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:18.847 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:18.848 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:18.848 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:18.848 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:18.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:19:18.884 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:19:18.908 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:18.904+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:18.989 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:18.989 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:18.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:18.989 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:18.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.989 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:19:18.989 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:18.990 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.450457450000002, 2024-09-13T09:19:18.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:19:18.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:18.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:18.990 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:18.990 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:18.990 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:18.990 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:19:18.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.990 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:18.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:19:18.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:18.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:18.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:19:18.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:19:18.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:18.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:19:18.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:19:18.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:18.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:19:18.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:19:18.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:18.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:19:18.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:19:18.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:18.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:19:18.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:19:18.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:18.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:19:18.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:19:18.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:18.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:19:18.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:19:18.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:18.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:19:18.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:19:18.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:18.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:19:18.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:19:18.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:18.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:18.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:18.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:19:18.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:19:18.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.999 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:18.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:18.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.000 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.000 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:19.000 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080890465000003, 2024-09-13T09:19:19.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:19:19.000 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.000 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.001 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.001 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.001 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:19:19.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.001 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:19.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:19.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:19:19.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:19:19.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:19:19.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:19:19.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:19:19.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:19:19.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:19:19.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:19:19.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:19:19.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:19:19.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:19:19.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.007 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.008 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.008 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:19.008 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080873144000002, 2024-09-13T09:19:19.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:19:19.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.009 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.009 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.009 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.009 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.009 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:19:19.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.010 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:19.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:19.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:19:19.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:19:19.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:19:19.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:19:19.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:19:19.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:19:19.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:19:19.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:19:19.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:19:19.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:19:19.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:19:19.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.016 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.017 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.017 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:19.018 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080856203000003, 2024-09-13T09:19:19.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:19:19.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.019 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.019 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:19:19.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.019 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:19.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:19.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:19:19.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:19:19.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:19:19.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:19:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:19:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:19:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:19:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:19:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:19:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:19:19.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:19:19.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.026 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.027 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.027 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:19.027 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080838974999999, 2024-09-13T09:19:19.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:19:19.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:19:19.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.028 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:19.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:19.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:19.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:19:19.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:19:19.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:19:19.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:19:19.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:19:19.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:19:19.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:19:19.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:19:19.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:19:19.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:19:19.035 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.035 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.035 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.036 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.036 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:19.036 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080802554000002, 2024-09-13T09:19:19.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:19:19.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:19:19.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.037 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:19.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:19.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:19.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:19:19.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:19:19.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:19:19.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:19:19.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:19:19.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:19:19.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:19:19.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:19:19.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:19:19.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:19:19.043 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.044 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.044 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.044 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:19.045 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080749091999998, 2024-09-13T09:19:19.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:19:19.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:19:19.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.046 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:19.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:19.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:19:19.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:19:19.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:19:19.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:19:19.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:19:19.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:19:19.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:19:19.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:19:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:19:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:19:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:19:19.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.052 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.052 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080727072000002, 2024-09-13T09:19:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:19:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:19:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:19:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:19:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:19:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:19:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:19:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080702436999999, 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:19:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:19:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:19:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:19:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:19:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080683913999998, 2024-09-13T09:19:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:19:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080666043000001, 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:19:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:19:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:19:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:19:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080648713000002, 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:19:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:19:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080631242999999, 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:19:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:19:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:19:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080614377000003, 2024-09-13T09:19:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:19:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:19:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:19:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080596589999999, 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.080552279999999, 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:19:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:19:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.847421791999999, 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:19:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:19:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:19:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.847404959000002, 2024-09-13T09:19:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:19:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:19:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.847388600999999, 2024-09-13T09:19:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:19:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:19:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:19:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:19:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:19:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:19:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:66592 3.7 3:e0817996:::rbd_data.1185b1e141e2.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 39.343028871000001, 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.051168777999999998, 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 66592 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643787+0000", 2024-09-13T09:19:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643799+0000", 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2140000000000001e-05 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643827+0000", 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8121999999999999e-05 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643841+0000", 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3969e-05 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643985+0000", 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014404199999999999 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.644371+0000", 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038588700000000001 2024-09-13T09:19:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.654317+0000", 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099452050000000004 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694641+0000", 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.040324802999999999 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694682+0000", 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0451999999999998e-05 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694828+0000", 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014580499999999999 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694958+0000", 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013029599999999999 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.112 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.112 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:19.112 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:19.112 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:19.113 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:18.981+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:19.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:19:19.114 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:19.115 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:19.115 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:19.115 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:19.115 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:19.116 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:19.116 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:19.108+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:19.130 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:19.131 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:19.151 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:19.170 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:19.189 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:19.256 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:19.252+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:19.256 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:19.257 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:19.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:19:19.322 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:19.317+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:19.417 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:19.409+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:19.424 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:19.424 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:19.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:19:19.517 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:19.512+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:19.611 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:19.611 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:19.618 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:19.616+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:19.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:19:19.719 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:19.712+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:19.751 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:19.751 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:19.766 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:19:19.820 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:19.816+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:19.907 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:19.920 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:19.913+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:19.922 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:19:20.022 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:20.017+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:20.086 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:20.086 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:20.101 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:19:20.123 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:20.116+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:20.224 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:20.217+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:20.261 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:20.261 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:20.275 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:19:20.362 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:20.361+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:20.425 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:20.424+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:20.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:19:20.549 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:20.549+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:20.627 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:20.625+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:20.628 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:20.628 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:20.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:19:20.729 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:20.728+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:20.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:19:20.822 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:20.823 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:20.829 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:20.829+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:20.839 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:19:20.931 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:20.924+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:21.010 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:21.010 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:21.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:19:21.070 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:21.069+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:21.132 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:21.128+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:21.151 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:21.151 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:21.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:19:21.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:19:21.256 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:21.252+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:21.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:19:21.352 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:21.348+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:21.355 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:21.355 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:21.372 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:21.372 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:21.389 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:21.389 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:21.406 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:21.406 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:21.426 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:21.435 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:21.428+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:21.447 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:21.448 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:21.467 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:21.467 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:21.469 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:21.469 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:21.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:19:21.490 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:21.491 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:21.536 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:21.533+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:21.631 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3:ed699f0f:::rbd_data.118591b29fad.000000000000008c:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:19:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.765518622000002, 2024-09-13T09:19:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94878089700000001, 2024-09-13T09:19:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:19:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:19:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:19:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:19:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:19:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810654+0000", 2024-09-13T09:19:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94834837599999999 2024-09-13T09:19:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810755+0000", 2024-09-13T09:19:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100987 2024-09-13T09:19:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810828+0000", 2024-09-13T09:19:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2751999999999997e-05 2024-09-13T09:19:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19465 3.10 3:0e3a5b30:::rbd_data.1185b72576e4.0000000000000062:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:19:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.679673885, 2024-09-13T09:19:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97489159000000003, 2024-09-13T09:19:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19465 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947890+0000", 2024-09-13T09:19:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947899+0000", 2024-09-13T09:19:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6840000000000008e-06 2024-09-13T09:19:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947917+0000", 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7731e-05 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947925+0000", 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4189999999999996e-06 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947990+0000", 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5909999999999997e-05 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947998+0000", 2024-09-13T09:19:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4449999999999999e-06 2024-09-13T09:19:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.919994+0000", 2024-09-13T09:19:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97199595900000002 2024-09-13T09:19:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920258+0000", 2024-09-13T09:19:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026439500000000001 2024-09-13T09:19:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920629+0000", 2024-09-13T09:19:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037096299999999998 2024-09-13T09:19:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.921797+0000", 2024-09-13T09:19:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001167721 2024-09-13T09:19:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922544+0000", 2024-09-13T09:19:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00074661200000000001 2024-09-13T09:19:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922628+0000", 2024-09-13T09:19:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4016000000000003e-05 2024-09-13T09:19:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922783+0000", 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015585499999999999 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51771 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882937411999997, 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083783107999999995, 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51771 2024-09-13T09:19:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744631+0000", 2024-09-13T09:19:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.943e-06 2024-09-13T09:19:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744639+0000", 2024-09-13T09:19:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:19:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744646+0000", 2024-09-13T09:19:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5860000000000001e-06 2024-09-13T09:19:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828151+0000", 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083505732999999999 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828286+0000", 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013457200000000001 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828411+0000", 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012564999999999999 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51772 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882914315999997, 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083939185999999999, 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51772 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744654+0000", 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2440000000000002e-06 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744662+0000", 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3659999999999992e-06 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744669+0000", 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6529999999999997e-06 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828455+0000", 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083785707000000001 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828512+0000", 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7549000000000001e-05 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828591+0000", 2024-09-13T09:19:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8274000000000003e-05 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51773 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882890660000001, 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084139096999999996, 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51773 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744674+0000", 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744677+0000", 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9399999999999998e-06 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744689+0000", 2024-09-13T09:19:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.153e-05 2024-09-13T09:19:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744705+0000", 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5679e-05 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828658+0000", 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083953190999999996 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828734+0000", 2024-09-13T09:19:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5784999999999994e-05 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828814+0000", 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579000000000002e-05 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51774 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882853298000001, 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084301713, 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51774 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744723+0000", 2024-09-13T09:19:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0832000000000001e-05 2024-09-13T09:19:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744733+0000", 2024-09-13T09:19:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031e-05 2024-09-13T09:19:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744740+0000", 2024-09-13T09:19:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0500000000000003e-06 2024-09-13T09:19:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828867+0000", 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084127417999999995 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828932+0000", 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4202000000000006e-05 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829014+0000", 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51775 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882819626, 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084465871999999997, 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51775 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744749+0000", 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5200000000000002e-06 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744757+0000", 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1100000000000003e-06 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744764+0000", 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8430000000000004e-06 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829058+0000", 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084293899000000005 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829111+0000", 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3315e-05 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829212+0000", 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010079200000000001 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51776 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882795913999999, 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084631102999999999, 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51776 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744769+0000", 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744772+0000", 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.14e-06 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744781+0000", 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.276e-06 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744788+0000", 2024-09-13T09:19:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0269999999999998e-06 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829254+0000", 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084466325999999994 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829310+0000", 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5852e-05 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829401+0000", 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1075000000000005e-05 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51777 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882740646000002, 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084901272, 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51777 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744824+0000", 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744828+0000", 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744840+0000", 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1997000000000001e-05 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744847+0000", 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0659999999999998e-06 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829460+0000", 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084613387999999998 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829555+0000", 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4999000000000003e-05 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829726+0000", 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017100899999999999 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51778 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882688201999997, 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085026334999999995, 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51778 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744877+0000", 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744881+0000", 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5999999999999998e-06 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744892+0000", 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1732999999999999e-05 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744899+0000", 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0759999999999997e-06 2024-09-13T09:19:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829772+0000", 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084872633000000003 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829826+0000", 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4330000000000003e-05 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829904+0000", 2024-09-13T09:19:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7558999999999994e-05 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51779 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882659633000003, 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085200608999999997, 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51779 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744905+0000", 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744909+0000", 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3500000000000001e-06 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744917+0000", 2024-09-13T09:19:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3129999999999999e-06 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744924+0000", 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6059999999999999e-06 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829976+0000", 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085051775999999996 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830030+0000", 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4404000000000003e-05 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830107+0000", 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6796000000000001e-05 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51780 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.687 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882636040000001, 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085367775000000007, 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51780 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744929+0000", 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744932+0000", 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744940+0000", 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1130000000000001e-06 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744947+0000", 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830148+0000", 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085201099000000002 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830205+0000", 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6623999999999999e-05 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830297+0000", 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2693000000000002e-05 2024-09-13T09:19:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51781 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882613004, 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085544856000000002, 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51781 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744952+0000", 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744955+0000", 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0960000000000001e-06 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745048+0000", 2024-09-13T09:19:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2806999999999994e-05 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745058+0000", 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0132999999999999e-05 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830338+0000", 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085279761999999995 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830420+0000", 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1551000000000006e-05 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830498+0000", 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8103999999999996e-05 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51782 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1794048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882501081999997, 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085605889000000004, 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51782 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745064+0000", 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745068+0000", 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.523e-06 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745076+0000", 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.354e-06 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745091+0000", 2024-09-13T09:19:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5274999999999999e-05 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830538+0000", 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085447238999999994 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830591+0000", 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2481000000000001e-05 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830671+0000", 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9716999999999996e-05 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51783 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1892352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882468436000003, 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085732662000000001, 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51783 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745100+0000", 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745108+0000", 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4360000000000002e-06 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745115+0000", 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9269999999999999e-06 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830707+0000", 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085591133999999999 2024-09-13T09:19:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830758+0000", 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1180000000000001e-05 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830830+0000", 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2268999999999995e-05 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51784 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.882436468000002, 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085902834999999997, 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51784 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745132+0000", 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.36e-06 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745143+0000", 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745150+0000", 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0400000000000004e-06 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830870+0000", 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085720086000000001 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830955+0000", 2024-09-13T09:19:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5321000000000005e-05 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831032+0000", 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6811999999999996e-05 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51785 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.881881573000001, 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085542517999999998, 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51785 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745160+0000", 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994755 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745697+0000", 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00053713000000000003 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745731+0000", 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3767999999999999e-05 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745754+0000", 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3748999999999999e-05 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831075+0000", 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085321067 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831130+0000", 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4156999999999997e-05 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831227+0000", 2024-09-13T09:19:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7093e-05 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51789 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.879188816999999, 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083633549000000001, 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51789 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748376+0000", 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748380+0000", 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7529999999999999e-06 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748392+0000", 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2376e-05 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748399+0000", 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0199999999999997e-06 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831866+0000", 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083466636999999996 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831933+0000", 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7144000000000006e-05 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832010+0000", 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7352000000000001e-05 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51790 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.879160251000002, 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083812485000000006, 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51790 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748405+0000", 2024-09-13T09:19:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748408+0000", 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3859999999999999e-06 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748417+0000", 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748424+0000", 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832055+0000", 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083631469 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832129+0000", 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4240999999999996e-05 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832218+0000", 2024-09-13T09:19:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8562999999999999e-05 2024-09-13T09:19:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51791 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2678784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.879135832000003, 2024-09-13T09:19:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083959632000000006, 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51791 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748429+0000", 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748432+0000", 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0800000000000002e-06 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748441+0000", 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9120000000000004e-06 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748470+0000", 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569e-05 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832260+0000", 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083790586 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832317+0000", 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6054999999999998e-05 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832390+0000", 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3033000000000004e-05 2024-09-13T09:19:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.729 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:21.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:19:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:19:21.731 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:21.724+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:21.792 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:19:21.793 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:19:21.793 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-13T09:19:21.793 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:19:21.793 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:19:21.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-13T09:19:21.868 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:21.864+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:21.908 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:81497 3.0 3:031c1c84:::rbd_object_map.118567e41306:head [call lock.assert_locked in=44b,call rbd.object_map_snap_remove in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:19:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:19:21.908058+0000", 2024-09-13T09:19:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.00023848900000000001, 2024-09-13T09:19:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027985299999999999, 2024-09-13T09:19:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 81497 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:19:21.908058+0000", 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:19:21.908058+0000", 2024-09-13T09:19:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:19:21.908057+0000", 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:19:21.908064+0000", 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0400000000000004e-06 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:19:21.908078+0000", 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3831e-05 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:19:21.908087+0000", 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5159999999999994e-06 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:19:21.908145+0000", 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8312e-05 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:19:21.908229+0000", 2024-09-13T09:19:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3985000000000003e-05 2024-09-13T09:19:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 1 2024-09-13T09:19:21.913 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:21.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:19:21.956 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:19:21.957 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:21.957 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:21.952+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:22.033 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:22.029+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:22.082 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:22.083 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:22.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:19:22.144 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:22.140+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:22.226 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:19:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.149160606999999, 2024-09-13T09:19:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:19:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:19:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:19:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:19:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56102 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5439488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.806319549999998, 2024-09-13T09:19:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060495140000000003, 2024-09-13T09:19:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56102 2024-09-13T09:19:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416845+0000", 2024-09-13T09:19:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416853+0000", 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416862+0000", 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1959999999999997e-06 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416868+0000", 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9719999999999997e-06 2024-09-13T09:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477088+0000", 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060219898000000001 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477154+0000", 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5605000000000006e-05 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477341+0000", 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018687599999999999 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56103 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5537792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.806291485999999, 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060576972999999999, 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56103 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416873+0000", 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416876+0000", 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.993e-06 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416883+0000", 2024-09-13T09:19:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.477e-06 2024-09-13T09:19:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416888+0000", 2024-09-13T09:19:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2580000000000003e-06 2024-09-13T09:19:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477357+0000", 2024-09-13T09:19:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060469055000000001 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477396+0000", 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9402e-05 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477451+0000", 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4262000000000003e-05 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56104 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5636096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.806272993999997, 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060778881999999999, 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56104 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416894+0000", 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4049999999999998e-06 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416900+0000", 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.959e-06 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416905+0000", 2024-09-13T09:19:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-13T09:19:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477465+0000", 2024-09-13T09:19:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060559460000000002 2024-09-13T09:19:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477521+0000", 2024-09-13T09:19:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5813000000000002e-05 2024-09-13T09:19:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477671+0000", 2024-09-13T09:19:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150366 2024-09-13T09:19:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56105 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5734400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.806228783000002, 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060898832, 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56105 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416939+0000", 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3550000000000001e-06 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416947+0000", 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0949999999999996e-06 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416953+0000", 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6790000000000002e-06 2024-09-13T09:19:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477708+0000", 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060755454 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477760+0000", 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1589999999999999e-05 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477835+0000", 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5394000000000004e-05 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56106 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5832704~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.806207466000004, 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061142999000000003, 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56106 2024-09-13T09:19:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416957+0000", 2024-09-13T09:19:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416960+0000", 2024-09-13T09:19:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6979999999999998e-06 2024-09-13T09:19:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416967+0000", 2024-09-13T09:19:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6919999999999996e-06 2024-09-13T09:19:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416973+0000", 2024-09-13T09:19:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5679999999999996e-06 2024-09-13T09:19:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477867+0000", 2024-09-13T09:19:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060894082000000002 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477924+0000", 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7015999999999998e-05 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478101+0000", 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017648300000000001 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56107 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5931008~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.805878851999999, 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060956639999999999, 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56107 2024-09-13T09:19:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416977+0000", 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996905 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417291+0000", 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031392799999999999 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417311+0000", 2024-09-13T09:19:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0343000000000002e-05 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417329+0000", 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7612e-05 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478123+0000", 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060794421000000001 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478174+0000", 2024-09-13T09:19:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0614e-05 2024-09-13T09:19:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478243+0000", 2024-09-13T09:19:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9226000000000004e-05 2024-09-13T09:19:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56108 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 6029312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.805819368000002, 2024-09-13T09:19:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061163618000000003, 2024-09-13T09:19:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56108 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417345+0000", 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417349+0000", 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7419999999999999e-06 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417367+0000", 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7714999999999999e-05 2024-09-13T09:19:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417382+0000", 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5396999999999998e-05 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478270+0000", 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060888022999999999 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478322+0000", 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2234999999999997e-05 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478509+0000", 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187109 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56121 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7307264~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.801835875000002, 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060455939, 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56121 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421332+0000", 2024-09-13T09:19:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:19:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421358+0000", 2024-09-13T09:19:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6427999999999998e-05 2024-09-13T09:19:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421374+0000", 2024-09-13T09:19:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6137000000000002e-05 2024-09-13T09:19:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481604+0000", 2024-09-13T09:19:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060229528999999997 2024-09-13T09:19:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481680+0000", 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6089999999999998e-05 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481785+0000", 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105259 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56122 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7405568~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.801762371999999, 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060604366999999999, 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56122 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421402+0000", 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421406+0000", 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.337e-06 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421415+0000", 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421434+0000", 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8617000000000001e-05 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481827+0000", 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060393883000000002 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481910+0000", 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2657999999999994e-05 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482007+0000", 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7057000000000003e-05 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56123 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7503872~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.801725834000003, 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060838125999999999, 2024-09-13T09:19:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56123 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421450+0000", 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1544e-05 2024-09-13T09:19:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421463+0000", 2024-09-13T09:19:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-13T09:19:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421468+0000", 2024-09-13T09:19:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6250000000000004e-06 2024-09-13T09:19:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482066+0000", 2024-09-13T09:19:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060597713999999997 2024-09-13T09:19:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482163+0000", 2024-09-13T09:19:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7112000000000001e-05 2024-09-13T09:19:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482277+0000", 2024-09-13T09:19:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114274 2024-09-13T09:19:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56124 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7602176~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.801691879000003, 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061054929000000001, 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56124 2024-09-13T09:19:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421476+0000", 2024-09-13T09:19:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2059999999999999e-06 2024-09-13T09:19:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421484+0000", 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7889999999999995e-06 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421489+0000", 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482321+0000", 2024-09-13T09:19:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060831477000000002 2024-09-13T09:19:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482403+0000", 2024-09-13T09:19:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2776999999999997e-05 2024-09-13T09:19:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482528+0000", 2024-09-13T09:19:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012491600000000001 2024-09-13T09:19:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56125 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7700480~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.801671581999997, 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061273721000000003, 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56125 2024-09-13T09:19:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421493+0000", 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421496+0000", 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.666e-06 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421503+0000", 2024-09-13T09:19:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:19:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421508+0000", 2024-09-13T09:19:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0919999999999997e-06 2024-09-13T09:19:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482575+0000", 2024-09-13T09:19:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061066339999999997 2024-09-13T09:19:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482659+0000", 2024-09-13T09:19:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4158000000000003e-05 2024-09-13T09:19:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482767+0000", 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010849 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56126 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7798784~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.801652216000001, 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061489349999999998, 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56126 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421531+0000", 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8218000000000001e-05 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421538+0000", 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.345e-06 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421544+0000", 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482813+0000", 2024-09-13T09:19:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061268959999999997 2024-09-13T09:19:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482900+0000", 2024-09-13T09:19:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6858000000000001e-05 2024-09-13T09:19:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483002+0000", 2024-09-13T09:19:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102599 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56127 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7897088~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.801616475000003, 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061729777, 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56127 2024-09-13T09:19:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421548+0000", 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421551+0000", 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421559+0000", 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4499999999999998e-06 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421564+0000", 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483067+0000", 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061503073999999998 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483153+0000", 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483278+0000", 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012522499999999999 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56319 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1613824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.795858729000003, 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060599663999999998, 2024-09-13T09:19:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56319 2024-09-13T09:19:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427309+0000", 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.672e-06 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427316+0000", 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6850000000000001e-06 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427322+0000", 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487756+0000", 2024-09-13T09:19:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060434817000000002 2024-09-13T09:19:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487825+0000", 2024-09-13T09:19:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8553000000000006e-05 2024-09-13T09:19:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487906+0000", 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1067000000000002e-05 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56320 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1712128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.795838914999997, 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060803930999999999, 2024-09-13T09:19:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56320 2024-09-13T09:19:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427328+0000", 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427379+0000", 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0343999999999998e-05 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427385+0000", 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3969999999999999e-06 2024-09-13T09:19:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487941+0000", 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060555805999999997 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488034+0000", 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488130+0000", 2024-09-13T09:19:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6348999999999995e-05 2024-09-13T09:19:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56321 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1810432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.795773978, 2024-09-13T09:19:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060999937999999997, 2024-09-13T09:19:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56321 2024-09-13T09:19:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427394+0000", 2024-09-13T09:19:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.721e-06 2024-09-13T09:19:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427403+0000", 2024-09-13T09:19:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7110000000000001e-06 2024-09-13T09:19:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427409+0000", 2024-09-13T09:19:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9710000000000001e-06 2024-09-13T09:19:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488183+0000", 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060774003 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488291+0000", 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010786899999999999 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488391+0000", 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100142 2024-09-13T09:19:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56322 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1908736~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.795427863, 2024-09-13T09:19:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060864109999999999, 2024-09-13T09:19:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56322 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427433+0000", 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996958 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427742+0000", 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000309084 2024-09-13T09:19:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427754+0000", 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1354e-05 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427761+0000", 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8059999999999995e-06 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488429+0000", 2024-09-13T09:19:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060667139000000002 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488496+0000", 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7595000000000006e-05 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488601+0000", 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105257 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56323 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 2007040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.794822168000003, 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060546618000000003, 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56323 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427767+0000", 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994245 2024-09-13T09:19:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428348+0000", 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00058032600000000004 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428360+0000", 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2103000000000001e-05 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428367+0000", 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1949999999999997e-06 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488650+0000", 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060282626999999998 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488745+0000", 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5365999999999995e-05 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488890+0000", 2024-09-13T09:19:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000144506 2024-09-13T09:19:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.330 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:22.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:19:22.332 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:22.328+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:22.433 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:22.429+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:22.521 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:19:22.534 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:22.533+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:22.635 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:22.633+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:22.647 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:22.647 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:22.662 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:19:22.735 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:22.728+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:22.792 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33548 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.798682819999996, 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025760904000000001, 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33548 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990124+0000", 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990130+0000", 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990135+0000", 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:19:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007826+0000", 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017691018999999999 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007896+0000", 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0059000000000001e-05 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.015883+0000", 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079875269999999995 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33751 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1593344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795263165999998, 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023779101, 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33751 2024-09-13T09:19:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993548+0000", 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.776e-06 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993555+0000", 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993560+0000", 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017188+0000", 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023627984000000001 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017271+0000", 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2798000000000002e-05 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017321+0000", 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0557999999999999e-05 2024-09-13T09:19:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33752 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795241046000001, 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023925173000000001, 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33752 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993566+0000", 2024-09-13T09:19:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:19:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993573+0000", 2024-09-13T09:19:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.759e-06 2024-09-13T09:19:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993578+0000", 2024-09-13T09:19:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8339999999999998e-06 2024-09-13T09:19:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017349+0000", 2024-09-13T09:19:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023770959000000001 2024-09-13T09:19:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017423+0000", 2024-09-13T09:19:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3652000000000006e-05 2024-09-13T09:19:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017489+0000", 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6748999999999994e-05 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33753 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1650688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795222563999999, 2024-09-13T09:19:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024066349000000001, 2024-09-13T09:19:22.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.809 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.809 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.809 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.810 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33753 2024-09-13T09:19:22.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.810 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993582+0000", 2024-09-13T09:19:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993585+0000", 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993591+0000", 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993595+0000", 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017515+0000", 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023920117000000001 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017583+0000", 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8485999999999994e-05 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017649+0000", 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33754 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795206269999994, 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02420106, 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33754 2024-09-13T09:19:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993601+0000", 2024-09-13T09:19:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:19:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993607+0000", 2024-09-13T09:19:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9669999999999998e-06 2024-09-13T09:19:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993611+0000", 2024-09-13T09:19:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:19:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017669+0000", 2024-09-13T09:19:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024057683999999999 2024-09-13T09:19:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017742+0000", 2024-09-13T09:19:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3397000000000002e-05 2024-09-13T09:19:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017800+0000", 2024-09-13T09:19:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7711999999999999e-05 2024-09-13T09:19:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33755 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1708032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795189851000004, 2024-09-13T09:19:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024346923999999999, 2024-09-13T09:19:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33755 2024-09-13T09:19:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993617+0000", 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993624+0000", 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0469999999999999e-06 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993628+0000", 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017830+0000", 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024201879999999999 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017897+0000", 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7096999999999998e-05 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017962+0000", 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5455000000000003e-05 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33756 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795173125000005, 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024485512000000001, 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33756 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993635+0000", 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993641+0000", 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993645+0000", 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2520000000000001e-06 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017979+0000", 2024-09-13T09:19:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024333950999999999 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018045+0000", 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5843e-05 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018118+0000", 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2417999999999997e-05 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33757 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1765376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795155726999994, 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024605578999999999, 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33757 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993649+0000", 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993651+0000", 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.841e-06 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993657+0000", 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993662+0000", 2024-09-13T09:19:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018139+0000", 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024476748 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018196+0000", 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7188000000000002e-05 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018255+0000", 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9045999999999997e-05 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33758 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795139094000007, 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024736898, 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33758 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993668+0000", 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993674+0000", 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993679+0000", 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.515e-06 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018284+0000", 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024604816000000002 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018356+0000", 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2782999999999998e-05 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018403+0000", 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6726e-05 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33759 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1822720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795122488000004, 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873635000000002, 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33759 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993682+0000", 2024-09-13T09:19:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993685+0000", 2024-09-13T09:19:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:19:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993691+0000", 2024-09-13T09:19:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1840000000000004e-06 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993696+0000", 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018429+0000", 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024733330000000001 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018494+0000", 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5233000000000004e-05 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018556+0000", 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2125999999999994e-05 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33760 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795105633000006, 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025058980000000002, 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33760 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993699+0000", 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993702+0000", 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.632e-06 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993708+0000", 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2280000000000003e-06 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993713+0000", 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018586+0000", 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873572 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018645+0000", 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9098000000000002e-05 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018759+0000", 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113224 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33761 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1880064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795088935999999, 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025229076, 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33761 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993718+0000", 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.092e-06 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993724+0000", 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0839999999999997e-06 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993729+0000", 2024-09-13T09:19:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:19:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018790+0000", 2024-09-13T09:19:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025061039 2024-09-13T09:19:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018871+0000", 2024-09-13T09:19:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0885999999999998e-05 2024-09-13T09:19:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018945+0000", 2024-09-13T09:19:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4773999999999999e-05 2024-09-13T09:19:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33762 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795072461999993, 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025559609, 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33762 2024-09-13T09:19:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993732+0000", 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993735+0000", 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993740+0000", 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993745+0000", 2024-09-13T09:19:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5290000000000002e-06 2024-09-13T09:19:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019043+0000", 2024-09-13T09:19:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025298355000000002 2024-09-13T09:19:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019134+0000", 2024-09-13T09:19:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0781999999999999e-05 2024-09-13T09:19:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019292+0000", 2024-09-13T09:19:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015829 2024-09-13T09:19:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33763 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795056521000006, 2024-09-13T09:19:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025732657999999999, 2024-09-13T09:19:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33763 2024-09-13T09:19:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993748+0000", 2024-09-13T09:19:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993755+0000", 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993762+0000", 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993767+0000", 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7230000000000004e-06 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019322+0000", 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025555564999999999 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019402+0000", 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9433999999999999e-05 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019481+0000", 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9739000000000003e-05 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33764 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795034146999996, 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025908526000000001, 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33764 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993773+0000", 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993779+0000", 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9610000000000002e-06 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993783+0000", 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.459e-06 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019513+0000", 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025729399 2024-09-13T09:19:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019609+0000", 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6504999999999998e-05 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019680+0000", 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0393000000000002e-05 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33765 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1994752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795017912000006, 2024-09-13T09:19:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026051009999999999, 2024-09-13T09:19:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33765 2024-09-13T09:19:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993789+0000", 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993795+0000", 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.088e-06 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993799+0000", 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019704+0000", 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025904311999999999 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019766+0000", 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2829000000000004e-05 2024-09-13T09:19:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019838+0000", 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1935999999999997e-05 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33766 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.795001857000003, 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026238971999999999, 2024-09-13T09:19:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33766 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993805+0000", 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:19:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993811+0000", 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9000000000000003e-06 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993816+0000", 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.527e-06 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019877+0000", 2024-09-13T09:19:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026061337 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019960+0000", 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3245999999999995e-05 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020042+0000", 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2053999999999997e-05 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33767 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.794985447000002, 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026407013999999999, 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33767 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993819+0000", 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993822+0000", 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993828+0000", 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993832+0000", 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.493e-06 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020072+0000", 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026239519999999999 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020151+0000", 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9278999999999998e-05 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020227+0000", 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5965999999999997e-05 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33768 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.794966994000006, 2024-09-13T09:19:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026564424, 2024-09-13T09:19:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33768 2024-09-13T09:19:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993840+0000", 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.996e-06 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993846+0000", 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993858+0000", 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2214e-05 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020253+0000", 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026394926999999999 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020329+0000", 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5193999999999999e-05 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020403+0000", 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4145000000000002e-05 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:78829 3.6 3:6418b130:::rbd_data.11853dd604c.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.572280800000001, 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041141776999999997, 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 78829 2024-09-13T09:19:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216522+0000", 2024-09-13T09:19:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216540+0000", 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8280000000000001e-05 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216577+0000", 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7151999999999999e-05 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216594+0000", 2024-09-13T09:19:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7113e-05 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216696+0000", 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010190400000000001 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216833+0000", 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136697 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.224657+0000", 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078241000000000005 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257365+0000", 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032707984000000002 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257404+0000", 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8979000000000001e-05 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257549+0000", 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000145737 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257666+0000", 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116754 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:22.892 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:22.894 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:19:22.894 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:22.888+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:22.994 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:22.988+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:23.072 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:23.073 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:23.073 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:23.073 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:23.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:19:23.095 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:23.093+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:23.196 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:23.192+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:23.258 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:23.258 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:23.258 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:23.258 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:23.258 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:23.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:19:23.316 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:23.312+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:23.398 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:23.392+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:23.437 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 95.898089874999997, 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:19:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:19:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:19:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:19:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:19:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:19:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:19:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:19:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528522890000005, 2024-09-13T09:19:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:19:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:19:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:19:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528505569000004, 2024-09-13T09:19:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:19:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:19:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528488628000005, 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:19:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:19:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:19:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:19:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:19:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:19:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528471400000001, 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:19:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528434978999996, 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528381517, 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:19:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:19:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:19:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:19:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:19:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:19:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528359496999997, 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:19:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:19:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:19:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:19:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:19:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:19:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528334861999994, 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:19:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:19:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:19:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:19:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:19:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:19:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:19:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528316339, 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:19:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.484 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.484 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:23.484 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528298468000003, 2024-09-13T09:19:23.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:19:23.484 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.484 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:19:23.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528281137999997, 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.488 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:19:23.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.489 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:23.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:19:23.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:19:23.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:19:23.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:19:23.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:19:23.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:19:23.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:19:23.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:19:23.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:19:23.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:19:23.492 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.492 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.492 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528263667999994, 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:19:23.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:19:23.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:19:23.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:19:23.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:19:23.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528246801999998, 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:23.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:19:23.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:19:23.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:19:23.501 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.501 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.501 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.501 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.501 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:23.501 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528229014999994, 2024-09-13T09:19:23.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:19:23.502 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.502 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.502 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:19:23.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.502 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:19:23.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:19:23.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:19:23.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:19:23.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:19:23.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:19:23.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:19:23.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:19:23.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:19:23.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:19:23.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.506 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.506 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.506 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:23.506 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.528184705000001, 2024-09-13T09:19:23.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:19:23.507 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.507 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:23.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:19:23.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.507 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:19:23.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:19:23.510 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.511 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.511 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.511 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:23.511 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.295054217000001, 2024-09-13T09:19:23.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:19:23.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:23.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.512 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:19:23.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.512 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:23.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:19:23.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:19:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:19:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:19:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:19:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.516 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.516 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:23.516 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.295037383999997, 2024-09-13T09:19:23.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:19:23.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:23.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.516 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:19:23.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.517 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:23.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:23.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:23.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:19:23.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:19:23.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:19:23.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:23.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:19:23.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:19:23.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:19:23.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:19:23.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.520 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.295021026000001, 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:19:23.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:19:23.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:19:23.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:19:23.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.525 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.525 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:66592 3.7 3:e0817996:::rbd_data.1185b1e141e2.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:19:23.525 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:23.525 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.790661296000003, 2024-09-13T09:19:23.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.051168777999999998, 2024-09-13T09:19:23.526 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:23.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:23.526 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:23.526 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:23.526 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:23.526 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 66592 2024-09-13T09:19:23.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.526 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:23.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:23.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643787+0000", 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643799+0000", 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2140000000000001e-05 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643827+0000", 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8121999999999999e-05 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643841+0000", 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3969e-05 2024-09-13T09:19:23.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643985+0000", 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014404199999999999 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.644371+0000", 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038588700000000001 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-13T09:19:23.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.654317+0000", 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099452050000000004 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694641+0000", 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.040324802999999999 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694682+0000", 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0451999999999998e-05 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694828+0000", 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014580499999999999 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694958+0000", 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013029599999999999 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:23.531 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:23.532 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:19:23.533 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:23.529+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:23.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:19:23.634 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:23.633+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:23.669 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:23.669 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:23.669 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:23.669 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:23.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:19:23.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:23.727 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:23.735 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:23.728+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:23.739 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:19:23.744 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:23.745 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:23.762 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:23.778 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:23.799 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:23.825 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:23.825 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:23.836 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:23.832+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:23.845 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:23.845 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:23.860 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:23.861 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:23.869 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:23.869 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:23.876 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:19:23.940 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:23.936+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:24.003 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:24.003 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:24.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:19:24.066 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:24.065+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:24.138 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:24.137+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:24.143 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:24.143 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:24.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:19:24.240 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:24.236+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:24.330 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:24.330 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:24.341 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:24.340+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:24.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:19:24.442 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:24.441+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:24.468 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:24.469 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:24.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:19:24.570 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:24.569+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:24.645 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:24.644+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:24.654 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:24.654 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:24.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:19:24.754 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:24.753+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:24.822 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:19:24.838 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:24.838 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:24.847 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:24.845+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:24.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:19:24.947 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:24.945+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:25.038 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:25.039 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:25.049 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:25.044+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:25.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:19:25.149 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:25.149+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:25.221 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:25.221 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:25.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:19:25.251 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:25.244+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:25.352 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:25.348+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:25.405 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:25.405 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:25.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:19:25.470 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:25.469+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:25.549 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:25.549 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:25.554 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:25.548+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:25.564 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:19:25.654 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:25.648+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:25.701 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:25.701 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:25.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:19:25.755 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:25.753+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:25.857 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:25.852+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:25.893 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:25.893 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:25.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:19:25.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:19:25.957 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:25.957+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:26.056 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:26.059 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:26.057+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:26.072 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:26.072 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:26.081 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:26.081 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:26.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:26.081 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:26.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.081 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:3614 3.17 3:ed699f0f:::rbd_data.118591b29fad.000000000000008c:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:19:26.081 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:26.081 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 103.21550203699999, 2024-09-13T09:19:26.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94878089700000001, 2024-09-13T09:19:26.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3614 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862047+0000", 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862044+0000", 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862060+0000", 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5460000000000001e-05 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862106+0000", 2024-09-13T09:19:26.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6279999999999997e-05 2024-09-13T09:19:26.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862135+0000", 2024-09-13T09:19:26.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8666e-05 2024-09-13T09:19:26.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862278+0000", 2024-09-13T09:19:26.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014272300000000001 2024-09-13T09:19:26.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:26.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:42.862306+0000", 2024-09-13T09:19:26.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8323e-05 2024-09-13T09:19:26.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810654+0000", 2024-09-13T09:19:26.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.94834837599999999 2024-09-13T09:19:26.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810755+0000", 2024-09-13T09:19:26.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100987 2024-09-13T09:19:26.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:43.810828+0000", 2024-09-13T09:19:26.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2751999999999997e-05 2024-09-13T09:19:26.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.086 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.087 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:19465 3.10 3:0e3a5b30:::rbd_data.1185b72576e4.0000000000000062:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:19:26.087 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:26.087 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 89.129657300000005, 2024-09-13T09:19:26.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97489159000000003, 2024-09-13T09:19:26.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.087 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:26.087 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.087 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.087 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.087 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 19465 2024-09-13T09:19:26.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.088 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:26.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947892+0000", 2024-09-13T09:19:26.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947890+0000", 2024-09-13T09:19:26.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:26.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947899+0000", 2024-09-13T09:19:26.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6840000000000008e-06 2024-09-13T09:19:26.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947917+0000", 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7731e-05 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947925+0000", 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4189999999999996e-06 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947990+0000", 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5909999999999997e-05 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:26.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:56.947998+0000", 2024-09-13T09:19:26.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4449999999999999e-06 2024-09-13T09:19:26.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.919994+0000", 2024-09-13T09:19:26.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.97199595900000002 2024-09-13T09:19:26.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920258+0000", 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026439500000000001 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.920629+0000", 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037096299999999998 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.921797+0000", 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001167721 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:26.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922544+0000", 2024-09-13T09:19:26.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00074661200000000001 2024-09-13T09:19:26.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:26.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922628+0000", 2024-09-13T09:19:26.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4016000000000003e-05 2024-09-13T09:19:26.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:57.922783+0000", 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015585499999999999 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51771 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332920827000002, 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083783107999999995, 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.094 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51771 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744628+0000", 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744631+0000", 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.943e-06 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744639+0000", 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744646+0000", 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5860000000000001e-06 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828151+0000", 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083505732999999999 2024-09-13T09:19:26.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828286+0000", 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013457200000000001 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828411+0000", 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012564999999999999 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51772 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332897731000003, 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083939185999999999, 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51772 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:26.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:26.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744651+0000", 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744654+0000", 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2440000000000002e-06 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744662+0000", 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3659999999999992e-06 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744669+0000", 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6529999999999997e-06 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828455+0000", 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083785707000000001 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828512+0000", 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7549000000000001e-05 2024-09-13T09:19:26.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828591+0000", 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8274000000000003e-05 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51773 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332874074999999, 2024-09-13T09:19:26.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084139096999999996, 2024-09-13T09:19:26.104 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.104 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.104 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.104 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.104 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.104 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51773 2024-09-13T09:19:26.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.104 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:26.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744675+0000", 2024-09-13T09:19:26.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744674+0000", 2024-09-13T09:19:26.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744677+0000", 2024-09-13T09:19:26.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9399999999999998e-06 2024-09-13T09:19:26.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744689+0000", 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.153e-05 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744705+0000", 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5679e-05 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828658+0000", 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083953190999999996 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828734+0000", 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5784999999999994e-05 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828814+0000", 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579000000000002e-05 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51774 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332836712999999, 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084301713, 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51774 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744712+0000", 2024-09-13T09:19:26.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744723+0000", 2024-09-13T09:19:26.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0832000000000001e-05 2024-09-13T09:19:26.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744733+0000", 2024-09-13T09:19:26.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031e-05 2024-09-13T09:19:26.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744740+0000", 2024-09-13T09:19:26.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0500000000000003e-06 2024-09-13T09:19:26.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828867+0000", 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084127417999999995 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.828932+0000", 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4202000000000006e-05 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829014+0000", 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51775 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332803040999998, 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084465871999999997, 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.114 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51775 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:26.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744746+0000", 2024-09-13T09:19:26.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744749+0000", 2024-09-13T09:19:26.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5200000000000002e-06 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744757+0000", 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1100000000000003e-06 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744764+0000", 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8430000000000004e-06 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829058+0000", 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084293899000000005 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829111+0000", 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3315e-05 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829212+0000", 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010079200000000001 2024-09-13T09:19:26.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51776 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332779328999997, 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084631102999999999, 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.119 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51776 2024-09-13T09:19:26.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.120 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:26.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744770+0000", 2024-09-13T09:19:26.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744769+0000", 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744772+0000", 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.14e-06 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744781+0000", 2024-09-13T09:19:26.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.276e-06 2024-09-13T09:19:26.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744788+0000", 2024-09-13T09:19:26.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0269999999999998e-06 2024-09-13T09:19:26.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829254+0000", 2024-09-13T09:19:26.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084466325999999994 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829310+0000", 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5852e-05 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829401+0000", 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1075000000000005e-05 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51777 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332724061, 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084901272, 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51777 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744825+0000", 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744824+0000", 2024-09-13T09:19:26.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744828+0000", 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744840+0000", 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1997000000000001e-05 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744847+0000", 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0659999999999998e-06 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829460+0000", 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084613387999999998 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829555+0000", 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4999000000000003e-05 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829726+0000", 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017100899999999999 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51778 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332671617000003, 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085026334999999995, 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.128 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.129 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.129 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.129 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.129 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51778 2024-09-13T09:19:26.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.129 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:26.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744878+0000", 2024-09-13T09:19:26.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744877+0000", 2024-09-13T09:19:26.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744881+0000", 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5999999999999998e-06 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744892+0000", 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1732999999999999e-05 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744899+0000", 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0759999999999997e-06 2024-09-13T09:19:26.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829772+0000", 2024-09-13T09:19:26.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.084872633000000003 2024-09-13T09:19:26.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829826+0000", 2024-09-13T09:19:26.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4330000000000003e-05 2024-09-13T09:19:26.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829904+0000", 2024-09-13T09:19:26.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7558999999999994e-05 2024-09-13T09:19:26.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.133 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.133 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51779 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.133 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:26.133 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332643048000001, 2024-09-13T09:19:26.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085200608999999997, 2024-09-13T09:19:26.133 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51779 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744906+0000", 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744905+0000", 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744909+0000", 2024-09-13T09:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3500000000000001e-06 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744917+0000", 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3129999999999999e-06 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744924+0000", 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6059999999999999e-06 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.829976+0000", 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085051775999999996 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830030+0000", 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4404000000000003e-05 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830107+0000", 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6796000000000001e-05 2024-09-13T09:19:26.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51780 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332619455, 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085367775000000007, 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.138 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51780 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744930+0000", 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744929+0000", 2024-09-13T09:19:26.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:26.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744932+0000", 2024-09-13T09:19:26.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:19:26.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744940+0000", 2024-09-13T09:19:26.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1130000000000001e-06 2024-09-13T09:19:26.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.141 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744947+0000", 2024-09-13T09:19:26.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:19:26.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.141 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830148+0000", 2024-09-13T09:19:26.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085201099000000002 2024-09-13T09:19:26.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830205+0000", 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6623999999999999e-05 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830297+0000", 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2693000000000002e-05 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51781 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332596418999998, 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085544856000000002, 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51781 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744953+0000", 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744952+0000", 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.744955+0000", 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0960000000000001e-06 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745048+0000", 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2806999999999994e-05 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745058+0000", 2024-09-13T09:19:26.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0132999999999999e-05 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830338+0000", 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085279761999999995 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830420+0000", 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1551000000000006e-05 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830498+0000", 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8103999999999996e-05 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51782 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1794048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332484497000003, 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085605889000000004, 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.147 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51782 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745065+0000", 2024-09-13T09:19:26.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745064+0000", 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745068+0000", 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.523e-06 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745076+0000", 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.354e-06 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745091+0000", 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5274999999999999e-05 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830538+0000", 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085447238999999994 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830591+0000", 2024-09-13T09:19:26.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2481000000000001e-05 2024-09-13T09:19:26.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830671+0000", 2024-09-13T09:19:26.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9716999999999996e-05 2024-09-13T09:19:26.151 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51783 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1892352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332451851000002, 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085732662000000001, 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.152 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51783 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745097+0000", 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745100+0000", 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745108+0000", 2024-09-13T09:19:26.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4360000000000002e-06 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745115+0000", 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9269999999999999e-06 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830707+0000", 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085591133999999999 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830758+0000", 2024-09-13T09:19:26.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1180000000000001e-05 2024-09-13T09:19:26.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830830+0000", 2024-09-13T09:19:26.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2268999999999995e-05 2024-09-13T09:19:26.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.156 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51784 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.332419883, 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085902834999999997, 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51784 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.157 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745129+0000", 2024-09-13T09:19:26.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745132+0000", 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.36e-06 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745143+0000", 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745150+0000", 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0400000000000004e-06 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830870+0000", 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085720086000000001 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.830955+0000", 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5321000000000005e-05 2024-09-13T09:19:26.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831032+0000", 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6811999999999996e-05 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51785 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.331864988, 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085542517999999998, 2024-09-13T09:19:26.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51785 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745684+0000", 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745160+0000", 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994755 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745697+0000", 2024-09-13T09:19:26.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00053713000000000003 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745731+0000", 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3767999999999999e-05 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.745754+0000", 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3748999999999999e-05 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831075+0000", 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.085321067 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831130+0000", 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4156999999999997e-05 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831227+0000", 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7093e-05 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.165 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.166 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.166 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51789 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.166 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:26.166 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.329172231999998, 2024-09-13T09:19:26.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083633549000000001, 2024-09-13T09:19:26.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51789 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748377+0000", 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748376+0000", 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748380+0000", 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7529999999999999e-06 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748392+0000", 2024-09-13T09:19:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2376e-05 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748399+0000", 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0199999999999997e-06 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831866+0000", 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083466636999999996 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.831933+0000", 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7144000000000006e-05 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832010+0000", 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7352000000000001e-05 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.170 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51790 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.329143666, 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083812485000000006, 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51790 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748406+0000", 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748405+0000", 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748408+0000", 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3859999999999999e-06 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748417+0000", 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748424+0000", 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832055+0000", 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083631469 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832129+0000", 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4240999999999996e-05 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832218+0000", 2024-09-13T09:19:26.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8562999999999999e-05 2024-09-13T09:19:26.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.175 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.175 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:51791 3.12 3:4b7410a9:::rbd_data.118568665621.000000000000000b:1d [read 2678784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.175 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:26.175 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.329119247000001, 2024-09-13T09:19:26.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083959632000000006, 2024-09-13T09:19:26.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 51791 2024-09-13T09:19:26.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.176 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:26.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748430+0000", 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748429+0000", 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748432+0000", 2024-09-13T09:19:26.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0800000000000002e-06 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748441+0000", 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9120000000000004e-06 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.748470+0000", 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569e-05 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832260+0000", 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.083790586 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832317+0000", 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6054999999999998e-05 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:34.832390+0000", 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3033000000000004e-05 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.179 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.180 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.180 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:26.181 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:26.182 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:19:26.182 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:26.183 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:26.183 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:26.183 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:26.183 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:26.184 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:26.184 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:26.184 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:26.185 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:26.176+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:26.191 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:26.285 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:26.280+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:26.312 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:26.312 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:26.312 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:26.313 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:26.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:19:26.412 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:26.408+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:26.487 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:26.484+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:26.491 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:26.491 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:26.491 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:26.491 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:26.491 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:26.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:19:26.592 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:26.588+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:26.679 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:26.679 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:26.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:26.679 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:26.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.679 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:19:26.679 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:26.679 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.601501317, 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:26.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:26.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:19:26.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:26.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:19:26.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:19:26.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:19:26.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:26.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:19:26.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:19:26.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:19:26.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:19:26.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:19:26.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:19:26.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:19:26.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:19:26.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:19:26.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:26.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:19:26.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:19:26.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:26.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:19:26.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:19:26.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:26.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:19:26.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:19:26.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:19:26.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:19:26.688 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.688 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.688 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.688 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56102 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5439488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.688 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:26.688 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.258660259999999, 2024-09-13T09:19:26.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060495140000000003, 2024-09-13T09:19:26.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.689 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.689 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.689 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.689 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.689 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56102 2024-09-13T09:19:26.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.689 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416845+0000", 2024-09-13T09:19:26.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416853+0000", 2024-09-13T09:19:26.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:19:26.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416862+0000", 2024-09-13T09:19:26.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1959999999999997e-06 2024-09-13T09:19:26.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416868+0000", 2024-09-13T09:19:26.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9719999999999997e-06 2024-09-13T09:19:26.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477088+0000", 2024-09-13T09:19:26.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060219898000000001 2024-09-13T09:19:26.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477154+0000", 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5605000000000006e-05 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477341+0000", 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018687599999999999 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.693 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56103 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5537792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.694 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:26.694 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.258632196000001, 2024-09-13T09:19:26.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060576972999999999, 2024-09-13T09:19:26.694 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.694 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.694 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.694 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.694 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.694 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56103 2024-09-13T09:19:26.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.694 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416873+0000", 2024-09-13T09:19:26.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416876+0000", 2024-09-13T09:19:26.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.993e-06 2024-09-13T09:19:26.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416883+0000", 2024-09-13T09:19:26.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.477e-06 2024-09-13T09:19:26.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416888+0000", 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2580000000000003e-06 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477357+0000", 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060469055000000001 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477396+0000", 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9402e-05 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477451+0000", 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4262000000000003e-05 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56104 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5636096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.698 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:26.699 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.258613703999998, 2024-09-13T09:19:26.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060778881999999999, 2024-09-13T09:19:26.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.699 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.699 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.699 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.699 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.700 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56104 2024-09-13T09:19:26.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.700 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:26.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:26.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:26.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416894+0000", 2024-09-13T09:19:26.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4049999999999998e-06 2024-09-13T09:19:26.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416900+0000", 2024-09-13T09:19:26.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.959e-06 2024-09-13T09:19:26.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416905+0000", 2024-09-13T09:19:26.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-13T09:19:26.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477465+0000", 2024-09-13T09:19:26.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060559460000000002 2024-09-13T09:19:26.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477521+0000", 2024-09-13T09:19:26.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5813000000000002e-05 2024-09-13T09:19:26.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477671+0000", 2024-09-13T09:19:26.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150366 2024-09-13T09:19:26.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.704 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.705 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56105 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5734400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.705 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:26.705 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.258569493000003, 2024-09-13T09:19:26.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060898832, 2024-09-13T09:19:26.705 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.705 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.705 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.705 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.705 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.705 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56105 2024-09-13T09:19:26.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.706 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:26.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:26.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416939+0000", 2024-09-13T09:19:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3550000000000001e-06 2024-09-13T09:19:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416947+0000", 2024-09-13T09:19:26.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0949999999999996e-06 2024-09-13T09:19:26.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416953+0000", 2024-09-13T09:19:26.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6790000000000002e-06 2024-09-13T09:19:26.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477708+0000", 2024-09-13T09:19:26.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060755454 2024-09-13T09:19:26.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477760+0000", 2024-09-13T09:19:26.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1589999999999999e-05 2024-09-13T09:19:26.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477835+0000", 2024-09-13T09:19:26.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5394000000000004e-05 2024-09-13T09:19:26.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.710 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56106 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5832704~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.710 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:26.710 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.258548175999998, 2024-09-13T09:19:26.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061142999000000003, 2024-09-13T09:19:26.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56106 2024-09-13T09:19:26.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.711 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:26.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416957+0000", 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416960+0000", 2024-09-13T09:19:26.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6979999999999998e-06 2024-09-13T09:19:26.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416967+0000", 2024-09-13T09:19:26.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6919999999999996e-06 2024-09-13T09:19:26.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416973+0000", 2024-09-13T09:19:26.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5679999999999996e-06 2024-09-13T09:19:26.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477867+0000", 2024-09-13T09:19:26.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060894082000000002 2024-09-13T09:19:26.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477924+0000", 2024-09-13T09:19:26.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7015999999999998e-05 2024-09-13T09:19:26.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478101+0000", 2024-09-13T09:19:26.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017648300000000001 2024-09-13T09:19:26.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.715 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.715 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56107 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5931008~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.715 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:26.715 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.258219562000001, 2024-09-13T09:19:26.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060956639999999999, 2024-09-13T09:19:26.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.716 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.716 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.716 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.716 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.716 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56107 2024-09-13T09:19:26.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.716 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:26.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:26.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416977+0000", 2024-09-13T09:19:26.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996905 2024-09-13T09:19:26.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417291+0000", 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031392799999999999 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417311+0000", 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0343000000000002e-05 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417329+0000", 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7612e-05 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478123+0000", 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060794421000000001 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478174+0000", 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0614e-05 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478243+0000", 2024-09-13T09:19:26.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9226000000000004e-05 2024-09-13T09:19:26.720 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.720 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.720 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.720 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56108 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 6029312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.720 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:26.720 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.258160078000003, 2024-09-13T09:19:26.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061163618000000003, 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56108 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:26.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417345+0000", 2024-09-13T09:19:26.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417349+0000", 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7419999999999999e-06 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417367+0000", 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7714999999999999e-05 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417382+0000", 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5396999999999998e-05 2024-09-13T09:19:26.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478270+0000", 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060888022999999999 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478322+0000", 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2234999999999997e-05 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478509+0000", 2024-09-13T09:19:26.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187109 2024-09-13T09:19:26.725 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.726 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.726 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.726 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56121 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7307264~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.726 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:26.726 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.254176585000003, 2024-09-13T09:19:26.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060455939, 2024-09-13T09:19:26.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.726 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.726 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.726 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56121 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421332+0000", 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421358+0000", 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6427999999999998e-05 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421374+0000", 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6137000000000002e-05 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481604+0000", 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060229528999999997 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481680+0000", 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6089999999999998e-05 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481785+0000", 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105259 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56122 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7405568~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.254103082, 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060604366999999999, 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56122 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421402+0000", 2024-09-13T09:19:26.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421406+0000", 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.337e-06 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421415+0000", 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421434+0000", 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8617000000000001e-05 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481827+0000", 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060393883000000002 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481910+0000", 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2657999999999994e-05 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482007+0000", 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7057000000000003e-05 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56123 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7503872~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.254066543999997, 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060838125999999999, 2024-09-13T09:19:26.735 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56123 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421450+0000", 2024-09-13T09:19:26.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1544e-05 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421463+0000", 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421468+0000", 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6250000000000004e-06 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482066+0000", 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060597713999999997 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482163+0000", 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7112000000000001e-05 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482277+0000", 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114274 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.739 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56124 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7602176~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.254032588999998, 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061054929000000001, 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.740 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56124 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421476+0000", 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2059999999999999e-06 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421484+0000", 2024-09-13T09:19:26.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7889999999999995e-06 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421489+0000", 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482321+0000", 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060831477000000002 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482403+0000", 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2776999999999997e-05 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482528+0000", 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012491600000000001 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56125 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7700480~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.744 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:26.745 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.254012291999999, 2024-09-13T09:19:26.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061273721000000003, 2024-09-13T09:19:26.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.745 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.745 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.745 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.745 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.745 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56125 2024-09-13T09:19:26.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421493+0000", 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421496+0000", 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.666e-06 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421503+0000", 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421508+0000", 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0919999999999997e-06 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482575+0000", 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061066339999999997 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482659+0000", 2024-09-13T09:19:26.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4158000000000003e-05 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482767+0000", 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010849 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56126 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7798784~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:26.749 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.253992926000002, 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061489349999999998, 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56126 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421531+0000", 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8218000000000001e-05 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421538+0000", 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.345e-06 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421544+0000", 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482813+0000", 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061268959999999997 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482900+0000", 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6858000000000001e-05 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483002+0000", 2024-09-13T09:19:26.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102599 2024-09-13T09:19:26.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.754 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.754 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56127 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7897088~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.754 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:26.754 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.253957184999997, 2024-09-13T09:19:26.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061729777, 2024-09-13T09:19:26.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56127 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421548+0000", 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421551+0000", 2024-09-13T09:19:26.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421559+0000", 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4499999999999998e-06 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421564+0000", 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483067+0000", 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061503073999999998 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483153+0000", 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483278+0000", 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012522499999999999 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.758 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56319 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1613824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.248199438999997, 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060599663999999998, 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.759 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56319 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427309+0000", 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.672e-06 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427316+0000", 2024-09-13T09:19:26.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6850000000000001e-06 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427322+0000", 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487756+0000", 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060434817000000002 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487825+0000", 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8553000000000006e-05 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487906+0000", 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1067000000000002e-05 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56320 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1712128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.763 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.248179624999999, 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060803930999999999, 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56320 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427328+0000", 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427379+0000", 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0343999999999998e-05 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427385+0000", 2024-09-13T09:19:26.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3969999999999999e-06 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487941+0000", 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060555805999999997 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488034+0000", 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488130+0000", 2024-09-13T09:19:26.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6348999999999995e-05 2024-09-13T09:19:26.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.768 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.768 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56321 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1810432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.768 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:26.768 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.248114688000001, 2024-09-13T09:19:26.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060999937999999997, 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56321 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427394+0000", 2024-09-13T09:19:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.721e-06 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427403+0000", 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7110000000000001e-06 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427409+0000", 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9710000000000001e-06 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488183+0000", 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060774003 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488291+0000", 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010786899999999999 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488391+0000", 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100142 2024-09-13T09:19:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56322 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1908736~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.247768573000002, 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060864109999999999, 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56322 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427433+0000", 2024-09-13T09:19:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996958 2024-09-13T09:19:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427742+0000", 2024-09-13T09:19:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000309084 2024-09-13T09:19:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427754+0000", 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1354e-05 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427761+0000", 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8059999999999995e-06 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488429+0000", 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060667139000000002 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488496+0000", 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7595000000000006e-05 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488601+0000", 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105257 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56323 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 2007040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.247162877999997, 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060546618000000003, 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56323 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427767+0000", 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994245 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428348+0000", 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00058032600000000004 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428360+0000", 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2103000000000001e-05 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428367+0000", 2024-09-13T09:19:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1949999999999997e-06 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488650+0000", 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060282626999999998 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488745+0000", 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5365999999999995e-05 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488890+0000", 2024-09-13T09:19:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000144506 2024-09-13T09:19:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:26.782 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:26.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:19:26.784 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:26.780+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:26.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:19:26.916 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:26.912+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:26.972 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:19:26.973 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:19:26.973 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T09:19:26.973 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.000000 2024-09-13T09:19:26.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0 2024-09-13T09:19:26.980 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:26.980 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:26.980 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:26.981 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:26.985 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:26.980+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:26.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:19:27.096 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:27.092+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:27.166 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:27.166 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:27.166 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:27.167 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:27.167 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:27.181 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:19:27.224 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:27.220+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:27.289 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:27.285+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:27.350 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:27.350 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:27.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:27.350 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33548 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.356551530999994, 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025760904000000001, 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33548 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.351 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:27.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990124+0000", 2024-09-13T09:19:27.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:19:27.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990130+0000", 2024-09-13T09:19:27.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:19:27.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990135+0000", 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007826+0000", 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017691018999999999 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007896+0000", 2024-09-13T09:19:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0059000000000001e-05 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.015883+0000", 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079875269999999995 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33751 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1593344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.353131876999996, 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023779101, 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33751 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993548+0000", 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.776e-06 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993555+0000", 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993560+0000", 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:19:27.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017188+0000", 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023627984000000001 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017271+0000", 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2798000000000002e-05 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017321+0000", 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0557999999999999e-05 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33752 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.353109756999999, 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023925173000000001, 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33752 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993566+0000", 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993573+0000", 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.759e-06 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993578+0000", 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8339999999999998e-06 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017349+0000", 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023770959000000001 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017423+0000", 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3652000000000006e-05 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017489+0000", 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6748999999999994e-05 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.364 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33753 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1650688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.353091274999997, 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024066349000000001, 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33753 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993582+0000", 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993585+0000", 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993591+0000", 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993595+0000", 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017515+0000", 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023920117000000001 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017583+0000", 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8485999999999994e-05 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017649+0000", 2024-09-13T09:19:27.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:19:27.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.369 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33754 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.369 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.353074981000006, 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02420106, 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33754 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:27.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993601+0000", 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993607+0000", 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9669999999999998e-06 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993611+0000", 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:19:27.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017669+0000", 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024057683999999999 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017742+0000", 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3397000000000002e-05 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017800+0000", 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7711999999999999e-05 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33755 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1708032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.353058562000001, 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024346923999999999, 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33755 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993617+0000", 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993624+0000", 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0469999999999999e-06 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993628+0000", 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017830+0000", 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024201879999999999 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017897+0000", 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7096999999999998e-05 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017962+0000", 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5455000000000003e-05 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.378 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33756 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.353041836000003, 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024485512000000001, 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33756 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993635+0000", 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993641+0000", 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993645+0000", 2024-09-13T09:19:27.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2520000000000001e-06 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017979+0000", 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024333950999999999 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018045+0000", 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5843e-05 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018118+0000", 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2417999999999997e-05 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33757 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1765376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.353024438000006, 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024605578999999999, 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.383 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33757 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:27.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993649+0000", 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993651+0000", 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.841e-06 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993657+0000", 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993662+0000", 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018139+0000", 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024476748 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018196+0000", 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7188000000000002e-05 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018255+0000", 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9045999999999997e-05 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.387 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33758 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.353007805000004, 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024736898, 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33758 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993668+0000", 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993674+0000", 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993679+0000", 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.515e-06 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018284+0000", 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024604816000000002 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018356+0000", 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2782999999999998e-05 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018403+0000", 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6726e-05 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33759 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1822720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.352991199000002, 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873635000000002, 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.392 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33759 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:27.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993682+0000", 2024-09-13T09:19:27.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993685+0000", 2024-09-13T09:19:27.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:19:27.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993691+0000", 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1840000000000004e-06 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993696+0000", 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018429+0000", 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024733330000000001 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018494+0000", 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5233000000000004e-05 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018556+0000", 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2125999999999994e-05 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.397 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33760 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.397 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:27.397 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.352974344000003, 2024-09-13T09:19:27.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025058980000000002, 2024-09-13T09:19:27.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.397 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.397 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.397 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.397 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.397 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33760 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993699+0000", 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993702+0000", 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.632e-06 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993708+0000", 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2280000000000003e-06 2024-09-13T09:19:27.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993713+0000", 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018586+0000", 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873572 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018645+0000", 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9098000000000002e-05 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018759+0000", 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113224 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33761 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1880064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.401 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.352957646999997, 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025229076, 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33761 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:27.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993718+0000", 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.092e-06 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993724+0000", 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0839999999999997e-06 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993729+0000", 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018790+0000", 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025061039 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018871+0000", 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0885999999999998e-05 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018945+0000", 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4773999999999999e-05 2024-09-13T09:19:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33762 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.352941173000005, 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025559609, 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33762 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993732+0000", 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993735+0000", 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993740+0000", 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:27.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993745+0000", 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5290000000000002e-06 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019043+0000", 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025298355000000002 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019134+0000", 2024-09-13T09:19:27.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0781999999999999e-05 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019292+0000", 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015829 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33763 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:27.410 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.352925232000004, 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025732657999999999, 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33763 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:27.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:27.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993748+0000", 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993755+0000", 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993762+0000", 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993767+0000", 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7230000000000004e-06 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019322+0000", 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025555564999999999 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019402+0000", 2024-09-13T09:19:27.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9433999999999999e-05 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019481+0000", 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9739000000000003e-05 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33764 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.352902857999993, 2024-09-13T09:19:27.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025908526000000001, 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33764 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993773+0000", 2024-09-13T09:19:27.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993779+0000", 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9610000000000002e-06 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993783+0000", 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.459e-06 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019513+0000", 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025729399 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019609+0000", 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6504999999999998e-05 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019680+0000", 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0393000000000002e-05 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33765 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1994752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.352886623000003, 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026051009999999999, 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33765 2024-09-13T09:19:27.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993789+0000", 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993795+0000", 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.088e-06 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993799+0000", 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019704+0000", 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025904311999999999 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019766+0000", 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2829000000000004e-05 2024-09-13T09:19:27.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019838+0000", 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1935999999999997e-05 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33766 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.352870568, 2024-09-13T09:19:27.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026238971999999999, 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33766 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993805+0000", 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:19:27.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993811+0000", 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9000000000000003e-06 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993816+0000", 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.527e-06 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019877+0000", 2024-09-13T09:19:27.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026061337 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019960+0000", 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3245999999999995e-05 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020042+0000", 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2053999999999997e-05 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.428 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33767 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.352854158, 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026407013999999999, 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33767 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.429 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993819+0000", 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993822+0000", 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993828+0000", 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993832+0000", 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.493e-06 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020072+0000", 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026239519999999999 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020151+0000", 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9278999999999998e-05 2024-09-13T09:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020227+0000", 2024-09-13T09:19:27.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5965999999999997e-05 2024-09-13T09:19:27.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.433 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.433 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33768 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.433 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.352835705000004, 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026564424, 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33768 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993840+0000", 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.996e-06 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993846+0000", 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993858+0000", 2024-09-13T09:19:27.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2214e-05 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020253+0000", 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026394926999999999 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020329+0000", 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5193999999999999e-05 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020403+0000", 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4145000000000002e-05 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:78829 3.6 3:6418b130:::rbd_data.11853dd604c.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.130149510999999, 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041141776999999997, 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:27.438 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 78829 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216522+0000", 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216540+0000", 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8280000000000001e-05 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216577+0000", 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7151999999999999e-05 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216594+0000", 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7113e-05 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216696+0000", 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010190400000000001 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216833+0000", 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136697 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.224657+0000", 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078241000000000005 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257365+0000", 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032707984000000002 2024-09-13T09:19:27.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257404+0000", 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8979000000000001e-05 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257549+0000", 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000145737 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257666+0000", 2024-09-13T09:19:27.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116754 2024-09-13T09:19:27.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.444 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.444 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.444 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:27.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:19:27.446 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:27.445+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:27.547 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:27.540+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:27.577 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:27.577 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:27.577 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:27.577 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:27.591 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:19:27.648 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:27.645+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:27.749 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:27.744+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:27.754 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:27.755 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:27.769 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:19:27.835 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.2 primary-affinity to 0 (802) 2024-09-13T09:19:27.839 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:19:27.850 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:27.844+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:27.951 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:27.952 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:27.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:27.952 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:27.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.952 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:19:27.952 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:27.952 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 100.413375066, 2024-09-13T09:19:27.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:19:27.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:27.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:19:27.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:27.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:19:27.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:19:27.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:27.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.958 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043808080999995, 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:19:27.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043790759999993, 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:19:27.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:19:27.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:19:27.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:19:27.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:19:27.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:19:27.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:19:27.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.967 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.967 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.967 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:27.967 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043773818999995, 2024-09-13T09:19:27.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:19:27.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:27.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:19:27.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:27.971 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043756591000005, 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:27.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:27.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:19:27.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.04372017, 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.976 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:19:27.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043666708000003, 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:27.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043644688000001, 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:19:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:19:27.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.988 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043620052999998, 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.993 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.993 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:27.993 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043601530000004, 2024-09-13T09:19:27.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:19:27.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:27.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:19:27.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:27.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:19:27.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:19:27.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:27.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:19:27.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:19:27.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:27.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:19:27.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:19:27.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.997 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:27.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:27.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.997 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:27.997 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:27.997 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043583659000006, 2024-09-13T09:19:27.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:19:27.997 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:19:27.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:28.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:19:28.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:19:28.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:28.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:19:28.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:19:28.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043566329000001, 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:28.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:28.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:28.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:19:28.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:19:28.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:28.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:19:28.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:19:28.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:19:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:19:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043548858999998, 2024-09-13T09:19:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:19:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:19:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043531993000002, 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:19:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:19:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:19:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:19:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:19:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:19:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043514205999998, 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:19:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:19:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:19:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:19:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:19:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.043469896000005, 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:19:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.810339408000004, 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:19:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:19:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:19:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.810322575000001, 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.810306217000004, 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:19:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:19:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:19:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:19:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:19:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:19:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:19:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:19:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:19:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:19:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:66592 3.7 3:e0817996:::rbd_data.1185b1e141e2.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.305946487, 2024-09-13T09:19:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.051168777999999998, 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 66592 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643787+0000", 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643799+0000", 2024-09-13T09:19:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2140000000000001e-05 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643827+0000", 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8121999999999999e-05 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643841+0000", 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3969e-05 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643985+0000", 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014404199999999999 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.644371+0000", 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038588700000000001 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.654317+0000", 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099452050000000004 2024-09-13T09:19:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694641+0000", 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.040324802999999999 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694682+0000", 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0451999999999998e-05 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694828+0000", 2024-09-13T09:19:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014580499999999999 2024-09-13T09:19:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694958+0000", 2024-09-13T09:19:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013029599999999999 2024-09-13T09:19:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:28.043 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:28.044 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:27.949+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:28.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:19:28.090 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:28.089+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:28.153 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:28.149+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:28.184 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:28.184 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:28.185 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:28.185 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:28.199 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:19:28.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:19:28.260 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:28.256+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:28.355 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:28.353+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:28.368 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:28.369 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:28.369 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:28.369 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:28.369 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:28.371 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:28.384 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:19:28.387 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:28.403 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:28.419 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:28.419 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:28.439 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:28.456 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:28.452+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:28.459 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:28.459 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:28.479 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:28.480 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:28.500 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:28.500 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:28.553 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:28.553 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:28.564 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:28.560+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:28.568 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:19:28.658 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:28.656+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:28.687 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:28.687 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:28.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:19:28.759 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:28.752+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:28.827 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:28.827 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:28.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:19:28.889 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:28.889+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:28.961 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:28.956+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:28.982 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:28.982 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:28.998 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:19:29.086 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:29.085+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:29.163 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:29.156+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:29.170 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:29.170 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:29.185 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:19:29.270 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:29.269+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:29.353 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:29.353 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:29.366 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:29.365+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:29.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:19:29.468 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:29.465+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:29.539 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:29.539 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:29.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:19:29.598 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:29.597+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:29.669 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:29.669+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:29.725 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:29.725 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:29.740 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:19:29.782 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:29.781+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:29.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:19:29.871 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:29.869+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:29.917 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:29.917 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:29.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:19:29.982 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:29.981+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:30.073 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:30.068+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:30.110 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:30.110 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:30.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:19:30.174 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:30.168+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:30.247 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:30.247 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:30.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:19:30.275 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:30.273+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:30.376 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:30.373+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:30.431 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:30.431 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:30.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:19:30.492 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:30.488+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:30.520 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:19:30.572 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:30.572 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:30.577 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:30.577+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:30.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:19:30.668 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:30.668 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:30.679 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:30.677+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:30.684 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:30.700 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:30.707 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:30.707 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:30.707 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:30.707 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:30.717 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:30.717 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:30.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:19:30.738 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:30.759 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:30.759 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:30.779 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:30.772+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:30.780 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:30.780 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:30.802 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:30.840 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:30.840 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:30.840 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:30.840 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:30.841 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:30.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:19:30.870 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-13T09:19:30.880 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:30.877+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:30.981 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:30.977+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:31.009 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:19:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.933147560999998, 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:19:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:19:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:19:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:19:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:19:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:19:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:19:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:19:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56102 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5439488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.590306503999997, 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060495140000000003, 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56102 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416845+0000", 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416853+0000", 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416862+0000", 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1959999999999997e-06 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416868+0000", 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9719999999999997e-06 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477088+0000", 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060219898000000001 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477154+0000", 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5605000000000006e-05 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477341+0000", 2024-09-13T09:19:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018687599999999999 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56103 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5537792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.590278439999999, 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060576972999999999, 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56103 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416873+0000", 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416876+0000", 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.993e-06 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416883+0000", 2024-09-13T09:19:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.477e-06 2024-09-13T09:19:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416888+0000", 2024-09-13T09:19:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2580000000000003e-06 2024-09-13T09:19:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477357+0000", 2024-09-13T09:19:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060469055000000001 2024-09-13T09:19:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477396+0000", 2024-09-13T09:19:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9402e-05 2024-09-13T09:19:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477451+0000", 2024-09-13T09:19:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4262000000000003e-05 2024-09-13T09:19:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56104 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5636096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.590259948000003, 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060778881999999999, 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56104 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416894+0000", 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4049999999999998e-06 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416900+0000", 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.959e-06 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416905+0000", 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477465+0000", 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060559460000000002 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477521+0000", 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5813000000000002e-05 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477671+0000", 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150366 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56105 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5734400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.590215737000001, 2024-09-13T09:19:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060898832, 2024-09-13T09:19:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56105 2024-09-13T09:19:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416939+0000", 2024-09-13T09:19:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3550000000000001e-06 2024-09-13T09:19:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416947+0000", 2024-09-13T09:19:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0949999999999996e-06 2024-09-13T09:19:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416953+0000", 2024-09-13T09:19:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6790000000000002e-06 2024-09-13T09:19:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477708+0000", 2024-09-13T09:19:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060755454 2024-09-13T09:19:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477760+0000", 2024-09-13T09:19:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1589999999999999e-05 2024-09-13T09:19:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477835+0000", 2024-09-13T09:19:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5394000000000004e-05 2024-09-13T09:19:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56106 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5832704~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.590194420000003, 2024-09-13T09:19:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061142999000000003, 2024-09-13T09:19:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56106 2024-09-13T09:19:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416957+0000", 2024-09-13T09:19:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416960+0000", 2024-09-13T09:19:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6979999999999998e-06 2024-09-13T09:19:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416967+0000", 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6919999999999996e-06 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416973+0000", 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5679999999999996e-06 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477867+0000", 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060894082000000002 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477924+0000", 2024-09-13T09:19:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7015999999999998e-05 2024-09-13T09:19:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478101+0000", 2024-09-13T09:19:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017648300000000001 2024-09-13T09:19:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56107 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5931008~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.589865805999999, 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060956639999999999, 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56107 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416977+0000", 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996905 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417291+0000", 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031392799999999999 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417311+0000", 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0343000000000002e-05 2024-09-13T09:19:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417329+0000", 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7612e-05 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478123+0000", 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060794421000000001 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478174+0000", 2024-09-13T09:19:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0614e-05 2024-09-13T09:19:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478243+0000", 2024-09-13T09:19:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9226000000000004e-05 2024-09-13T09:19:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56108 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 6029312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.589806322000001, 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061163618000000003, 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56108 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417345+0000", 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417349+0000", 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7419999999999999e-06 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417367+0000", 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7714999999999999e-05 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417382+0000", 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5396999999999998e-05 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478270+0000", 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060888022999999999 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478322+0000", 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2234999999999997e-05 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478509+0000", 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187109 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56121 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7307264~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.585822829000001, 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060455939, 2024-09-13T09:19:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56121 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421332+0000", 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421358+0000", 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6427999999999998e-05 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421374+0000", 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6137000000000002e-05 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481604+0000", 2024-09-13T09:19:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060229528999999997 2024-09-13T09:19:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481680+0000", 2024-09-13T09:19:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6089999999999998e-05 2024-09-13T09:19:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481785+0000", 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105259 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56122 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7405568~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.585749325999998, 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060604366999999999, 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56122 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421402+0000", 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421406+0000", 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.337e-06 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421415+0000", 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421434+0000", 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8617000000000001e-05 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481827+0000", 2024-09-13T09:19:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060393883000000002 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481910+0000", 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2657999999999994e-05 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482007+0000", 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7057000000000003e-05 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56123 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7503872~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.585712788000002, 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060838125999999999, 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56123 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421450+0000", 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1544e-05 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421463+0000", 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421468+0000", 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6250000000000004e-06 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482066+0000", 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060597713999999997 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482163+0000", 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7112000000000001e-05 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482277+0000", 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114274 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56124 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7602176~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.585678833000003, 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061054929000000001, 2024-09-13T09:19:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56124 2024-09-13T09:19:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421476+0000", 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2059999999999999e-06 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421484+0000", 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7889999999999995e-06 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421489+0000", 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482321+0000", 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060831477000000002 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482403+0000", 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2776999999999997e-05 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482528+0000", 2024-09-13T09:19:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012491600000000001 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56125 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7700480~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.585658535999997, 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061273721000000003, 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56125 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421493+0000", 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421496+0000", 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.666e-06 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421503+0000", 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421508+0000", 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0919999999999997e-06 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482575+0000", 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061066339999999997 2024-09-13T09:19:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482659+0000", 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4158000000000003e-05 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482767+0000", 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010849 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56126 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7798784~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.58563917, 2024-09-13T09:19:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061489349999999998, 2024-09-13T09:19:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56126 2024-09-13T09:19:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421531+0000", 2024-09-13T09:19:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8218000000000001e-05 2024-09-13T09:19:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421538+0000", 2024-09-13T09:19:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.345e-06 2024-09-13T09:19:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421544+0000", 2024-09-13T09:19:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:19:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482813+0000", 2024-09-13T09:19:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061268959999999997 2024-09-13T09:19:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482900+0000", 2024-09-13T09:19:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6858000000000001e-05 2024-09-13T09:19:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483002+0000", 2024-09-13T09:19:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102599 2024-09-13T09:19:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56127 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7897088~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.585603429000003, 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061729777, 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56127 2024-09-13T09:19:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421548+0000", 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421551+0000", 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421559+0000", 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4499999999999998e-06 2024-09-13T09:19:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421564+0000", 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483067+0000", 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061503073999999998 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483153+0000", 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483278+0000", 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012522499999999999 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56319 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1613824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.579845683000002, 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060599663999999998, 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56319 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427309+0000", 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.672e-06 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427316+0000", 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6850000000000001e-06 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427322+0000", 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487756+0000", 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060434817000000002 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487825+0000", 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8553000000000006e-05 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487906+0000", 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1067000000000002e-05 2024-09-13T09:19:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56320 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1712128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.579825868999997, 2024-09-13T09:19:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060803930999999999, 2024-09-13T09:19:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56320 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427328+0000", 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-13T09:19:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427379+0000", 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0343999999999998e-05 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427385+0000", 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3969999999999999e-06 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487941+0000", 2024-09-13T09:19:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060555805999999997 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488034+0000", 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488130+0000", 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6348999999999995e-05 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56321 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1810432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.579760931999999, 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060999937999999997, 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56321 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427394+0000", 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.721e-06 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427403+0000", 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7110000000000001e-06 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427409+0000", 2024-09-13T09:19:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9710000000000001e-06 2024-09-13T09:19:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488183+0000", 2024-09-13T09:19:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060774003 2024-09-13T09:19:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488291+0000", 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010786899999999999 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488391+0000", 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100142 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56322 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1908736~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.579414817, 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060864109999999999, 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56322 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427433+0000", 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996958 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427742+0000", 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000309084 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427754+0000", 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1354e-05 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427761+0000", 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8059999999999995e-06 2024-09-13T09:19:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488429+0000", 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060667139000000002 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488496+0000", 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7595000000000006e-05 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488601+0000", 2024-09-13T09:19:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105257 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56323 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 2007040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.578809122000003, 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060546618000000003, 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56323 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427767+0000", 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994245 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428348+0000", 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00058032600000000004 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428360+0000", 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2103000000000001e-05 2024-09-13T09:19:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428367+0000", 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1949999999999997e-06 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488650+0000", 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060282626999999998 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488745+0000", 2024-09-13T09:19:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5365999999999995e-05 2024-09-13T09:19:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488890+0000", 2024-09-13T09:19:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000144506 2024-09-13T09:19:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.106 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:31.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:19:31.108 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:31.104+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:31.209 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:31.204+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:31.284 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:31.284 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:31.284 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:31.284 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:31.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:19:31.340 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:31.336+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:31.411 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:31.404+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:31.468 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:31.469 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:31.469 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:31.469 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:31.469 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:31.483 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:19:31.533 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:31.528+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:31.613 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:31.613+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:31.664 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33548 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.670519971999994, 2024-09-13T09:19:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025760904000000001, 2024-09-13T09:19:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33548 2024-09-13T09:19:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990124+0000", 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990130+0000", 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990135+0000", 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007826+0000", 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017691018999999999 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007896+0000", 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0059000000000001e-05 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.015883+0000", 2024-09-13T09:19:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079875269999999995 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33751 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1593344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.667100317999996, 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023779101, 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33751 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993548+0000", 2024-09-13T09:19:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.776e-06 2024-09-13T09:19:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993555+0000", 2024-09-13T09:19:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:19:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993560+0000", 2024-09-13T09:19:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017188+0000", 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023627984000000001 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017271+0000", 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2798000000000002e-05 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017321+0000", 2024-09-13T09:19:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0557999999999999e-05 2024-09-13T09:19:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33752 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.667078197999999, 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023925173000000001, 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33752 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993566+0000", 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993573+0000", 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.759e-06 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993578+0000", 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8339999999999998e-06 2024-09-13T09:19:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017349+0000", 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023770959000000001 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017423+0000", 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3652000000000006e-05 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017489+0000", 2024-09-13T09:19:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6748999999999994e-05 2024-09-13T09:19:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33753 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1650688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.667059715999997, 2024-09-13T09:19:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024066349000000001, 2024-09-13T09:19:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33753 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993582+0000", 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993585+0000", 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993591+0000", 2024-09-13T09:19:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:19:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993595+0000", 2024-09-13T09:19:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:19:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017515+0000", 2024-09-13T09:19:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023920117000000001 2024-09-13T09:19:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017583+0000", 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8485999999999994e-05 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017649+0000", 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33754 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.667043422000006, 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02420106, 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33754 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993601+0000", 2024-09-13T09:19:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:19:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993607+0000", 2024-09-13T09:19:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9669999999999998e-06 2024-09-13T09:19:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993611+0000", 2024-09-13T09:19:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:19:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017669+0000", 2024-09-13T09:19:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024057683999999999 2024-09-13T09:19:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017742+0000", 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3397000000000002e-05 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017800+0000", 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7711999999999999e-05 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33755 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1708032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.667027003000001, 2024-09-13T09:19:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024346923999999999, 2024-09-13T09:19:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33755 2024-09-13T09:19:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993617+0000", 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993624+0000", 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0469999999999999e-06 2024-09-13T09:19:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993628+0000", 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017830+0000", 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024201879999999999 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017897+0000", 2024-09-13T09:19:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7096999999999998e-05 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017962+0000", 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5455000000000003e-05 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33756 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.667010277000003, 2024-09-13T09:19:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024485512000000001, 2024-09-13T09:19:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33756 2024-09-13T09:19:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993635+0000", 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993641+0000", 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993645+0000", 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2520000000000001e-06 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017979+0000", 2024-09-13T09:19:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024333950999999999 2024-09-13T09:19:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018045+0000", 2024-09-13T09:19:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5843e-05 2024-09-13T09:19:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018118+0000", 2024-09-13T09:19:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2417999999999997e-05 2024-09-13T09:19:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33757 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1765376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.666992879000006, 2024-09-13T09:19:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024605578999999999, 2024-09-13T09:19:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33757 2024-09-13T09:19:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993649+0000", 2024-09-13T09:19:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993651+0000", 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.841e-06 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993657+0000", 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993662+0000", 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018139+0000", 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024476748 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018196+0000", 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7188000000000002e-05 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018255+0000", 2024-09-13T09:19:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9045999999999997e-05 2024-09-13T09:19:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33758 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.666976246000004, 2024-09-13T09:19:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024736898, 2024-09-13T09:19:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33758 2024-09-13T09:19:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993668+0000", 2024-09-13T09:19:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:19:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993674+0000", 2024-09-13T09:19:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:19:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993679+0000", 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.515e-06 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018284+0000", 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024604816000000002 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018356+0000", 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2782999999999998e-05 2024-09-13T09:19:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018403+0000", 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6726e-05 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33759 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1822720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.666959640000002, 2024-09-13T09:19:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873635000000002, 2024-09-13T09:19:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33759 2024-09-13T09:19:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993682+0000", 2024-09-13T09:19:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993685+0000", 2024-09-13T09:19:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:19:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993691+0000", 2024-09-13T09:19:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1840000000000004e-06 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993696+0000", 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018429+0000", 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024733330000000001 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018494+0000", 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5233000000000004e-05 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018556+0000", 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2125999999999994e-05 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33760 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.666942785000003, 2024-09-13T09:19:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025058980000000002, 2024-09-13T09:19:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33760 2024-09-13T09:19:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993699+0000", 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993702+0000", 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.632e-06 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993708+0000", 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2280000000000003e-06 2024-09-13T09:19:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993713+0000", 2024-09-13T09:19:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:19:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018586+0000", 2024-09-13T09:19:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873572 2024-09-13T09:19:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018645+0000", 2024-09-13T09:19:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9098000000000002e-05 2024-09-13T09:19:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018759+0000", 2024-09-13T09:19:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113224 2024-09-13T09:19:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33761 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1880064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.666926087999997, 2024-09-13T09:19:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025229076, 2024-09-13T09:19:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33761 2024-09-13T09:19:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993718+0000", 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.092e-06 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993724+0000", 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0839999999999997e-06 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993729+0000", 2024-09-13T09:19:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018790+0000", 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025061039 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018871+0000", 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0885999999999998e-05 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018945+0000", 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4773999999999999e-05 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33762 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.666909614000005, 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025559609, 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33762 2024-09-13T09:19:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993732+0000", 2024-09-13T09:19:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993735+0000", 2024-09-13T09:19:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:19:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993740+0000", 2024-09-13T09:19:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993745+0000", 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5290000000000002e-06 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019043+0000", 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025298355000000002 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019134+0000", 2024-09-13T09:19:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0781999999999999e-05 2024-09-13T09:19:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019292+0000", 2024-09-13T09:19:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015829 2024-09-13T09:19:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33763 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.666893673000004, 2024-09-13T09:19:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025732657999999999, 2024-09-13T09:19:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33763 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993748+0000", 2024-09-13T09:19:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993755+0000", 2024-09-13T09:19:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:19:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993762+0000", 2024-09-13T09:19:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:19:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993767+0000", 2024-09-13T09:19:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7230000000000004e-06 2024-09-13T09:19:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019322+0000", 2024-09-13T09:19:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025555564999999999 2024-09-13T09:19:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019402+0000", 2024-09-13T09:19:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9433999999999999e-05 2024-09-13T09:19:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019481+0000", 2024-09-13T09:19:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9739000000000003e-05 2024-09-13T09:19:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33764 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.666871298999993, 2024-09-13T09:19:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025908526000000001, 2024-09-13T09:19:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33764 2024-09-13T09:19:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993773+0000", 2024-09-13T09:19:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:19:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993779+0000", 2024-09-13T09:19:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9610000000000002e-06 2024-09-13T09:19:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993783+0000", 2024-09-13T09:19:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.459e-06 2024-09-13T09:19:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019513+0000", 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025729399 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019609+0000", 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6504999999999998e-05 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019680+0000", 2024-09-13T09:19:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0393000000000002e-05 2024-09-13T09:19:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33765 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1994752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.666855064000003, 2024-09-13T09:19:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026051009999999999, 2024-09-13T09:19:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33765 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993789+0000", 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993795+0000", 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.088e-06 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993799+0000", 2024-09-13T09:19:31.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:19:31.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019704+0000", 2024-09-13T09:19:31.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025904311999999999 2024-09-13T09:19:31.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019766+0000", 2024-09-13T09:19:31.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2829000000000004e-05 2024-09-13T09:19:31.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019838+0000", 2024-09-13T09:19:31.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1935999999999997e-05 2024-09-13T09:19:31.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.752 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.752 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33766 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.753 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:31.753 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.666839009, 2024-09-13T09:19:31.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026238971999999999, 2024-09-13T09:19:31.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.753 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.753 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.753 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.753 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.753 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33766 2024-09-13T09:19:31.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.753 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:31.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993805+0000", 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993811+0000", 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9000000000000003e-06 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993816+0000", 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.527e-06 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019877+0000", 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026061337 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019960+0000", 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3245999999999995e-05 2024-09-13T09:19:31.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020042+0000", 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2053999999999997e-05 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33767 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.666822599, 2024-09-13T09:19:31.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026407013999999999, 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33767 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:31.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993819+0000", 2024-09-13T09:19:31.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993822+0000", 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993828+0000", 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993832+0000", 2024-09-13T09:19:31.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.493e-06 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020072+0000", 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026239519999999999 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020151+0000", 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9278999999999998e-05 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020227+0000", 2024-09-13T09:19:31.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5965999999999997e-05 2024-09-13T09:19:31.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.762 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.762 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33768 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:31.763 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:31.763 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.666804146000004, 2024-09-13T09:19:31.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026564424, 2024-09-13T09:19:31.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.763 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:31.763 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.763 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.764 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33768 2024-09-13T09:19:31.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.764 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:31.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993840+0000", 2024-09-13T09:19:31.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.996e-06 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993846+0000", 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993858+0000", 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2214e-05 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020253+0000", 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026394926999999999 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020329+0000", 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5193999999999999e-05 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020403+0000", 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4145000000000002e-05 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.767 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.768 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:78829 3.6 3:6418b130:::rbd_data.11853dd604c.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:31.768 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:31.768 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.444117951999999, 2024-09-13T09:19:31.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041141776999999997, 2024-09-13T09:19:31.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:31.768 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:31.768 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:31.768 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:31.768 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 78829 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:31.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216522+0000", 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216540+0000", 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8280000000000001e-05 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216577+0000", 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7151999999999999e-05 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216594+0000", 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7113e-05 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216696+0000", 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010190400000000001 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:31.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216833+0000", 2024-09-13T09:19:31.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136697 2024-09-13T09:19:31.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:31.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.224657+0000", 2024-09-13T09:19:31.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078241000000000005 2024-09-13T09:19:31.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:31.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257365+0000", 2024-09-13T09:19:31.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032707984000000002 2024-09-13T09:19:31.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:31.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257404+0000", 2024-09-13T09:19:31.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8979000000000001e-05 2024-09-13T09:19:31.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:31.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257549+0000", 2024-09-13T09:19:31.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000145737 2024-09-13T09:19:31.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:31.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:31.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:31.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257666+0000", 2024-09-13T09:19:31.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116754 2024-09-13T09:19:31.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.775 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:31.775 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:31.775 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:31.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:19:31.777 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:31.772+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:31.878 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:31.872+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:31.963 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:31.964 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:31.978 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:31.972+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:31.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:19:32.079 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:32.077+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:32.152 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:32.152 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:32.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:19:32.180 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:32.177+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:32.281 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:32.277+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:32.296 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:19:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 104.75588689999999, 2024-09-13T09:19:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:19:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:19:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:19:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:19:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:19:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:19:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:19:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:19:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:19:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:19:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:19:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:19:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:19:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:19:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:19:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:19:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:19:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:19:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:19:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:19:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:19:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:19:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:19:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386319915000001, 2024-09-13T09:19:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:19:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:19:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:19:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:19:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:19:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:19:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:19:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:19:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386302594, 2024-09-13T09:19:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:19:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:19:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:19:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:19:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:19:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:19:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:19:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:19:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:19:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:19:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:19:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:19:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:19:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386285653000002, 2024-09-13T09:19:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:19:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:19:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:19:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:19:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:19:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:19:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386268424999997, 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:19:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:19:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:19:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:19:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:19:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386232003999993, 2024-09-13T09:19:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:19:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:19:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386178541999996, 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:19:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:19:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386156521999993, 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:19:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386131887000005, 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386113363999996, 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:19:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:19:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:19:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:19:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:19:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:19:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386095492999999, 2024-09-13T09:19:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:19:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:19:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:19:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:19:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:19:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:19:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:19:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:19:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386078162999993, 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:19:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:19:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:19:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386060693000005, 2024-09-13T09:19:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386043826999995, 2024-09-13T09:19:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:19:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:19:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:19:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:19:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:19:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:19:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:19:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.386026040000004, 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:19:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:19:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:19:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:19:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:19:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:19:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.385981729999997, 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:19:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:19:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:19:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:19:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:19:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:19:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.152851241999997, 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:19:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:19:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:19:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:19:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.152834408999993, 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:19:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:19:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 75.152818050999997, 2024-09-13T09:19:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:19:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:19:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:19:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:66592 3.7 3:e0817996:::rbd_data.1185b1e141e2.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:19:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.648458321, 2024-09-13T09:19:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.051168777999999998, 2024-09-13T09:19:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 66592 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643787+0000", 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643799+0000", 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2140000000000001e-05 2024-09-13T09:19:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643827+0000", 2024-09-13T09:19:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8121999999999999e-05 2024-09-13T09:19:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643841+0000", 2024-09-13T09:19:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3969e-05 2024-09-13T09:19:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643985+0000", 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014404199999999999 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.644371+0000", 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038588700000000001 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.654317+0000", 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099452050000000004 2024-09-13T09:19:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694641+0000", 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.040324802999999999 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694682+0000", 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0451999999999998e-05 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694828+0000", 2024-09-13T09:19:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014580499999999999 2024-09-13T09:19:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694958+0000", 2024-09-13T09:19:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013029599999999999 2024-09-13T09:19:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:32.395 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:32.397 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:19:32.397 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:32.396+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:32.498 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:32.497+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:32.534 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:32.534 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:32.534 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:32.534 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:32.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:19:32.634 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:32.633+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:32.700 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:32.700+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:32.707 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:32.707 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:32.708 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:32.722 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:19:32.801 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:32.796+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:32.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:19:32.868 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:19:32.869 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:19:32.869 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T09:19:32.869 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.925232 2024-09-13T09:19:32.869 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0.9252318441129538 2024-09-13T09:19:32.893 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:32.893 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:32.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:32.893 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:32.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.893 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.901506764, 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.90145802799999, 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:19:32.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:19:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.902 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.901390359, 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:19:32.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:19:32.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.90125015, 2024-09-13T09:19:32.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:19:32.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:19:32.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:19:32.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:19:32.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:19:32.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.901159284, 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:32.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:19:32.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:19:32.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.901084424, 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.917 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.921 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.901013248, 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:19:32.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:19:32.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:19:32.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:19:32.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:19:32.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:19:32.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:19:32.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.900964195, 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.927 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:19:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.90090245899999, 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:19:32.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.932 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:32.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:19:32.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.935 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.900852789, 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:19:32.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:19:32.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:19:32.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:19:32.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.940 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:32.941 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.90080426999999, 2024-09-13T09:19:32.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:19:32.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:19:32.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.941 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:19:32.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:19:32.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:19:32.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:19:32.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:19:32.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:19:32.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.900755874, 2024-09-13T09:19:32.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:19:32.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.950 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.900708587, 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:19:32.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:19:32.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:19:32.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:19:32.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:19:32.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:19:32.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:19:32.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:19:32.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:19:32.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.955 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.955 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.900625185, 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:19:32.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:19:32.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:19:32.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:19:32.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:19:32.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:32.960 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.900551459, 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:19:32.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.964 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.965 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.965 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.965 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:32.965 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.90048333599999, 2024-09-13T09:19:32.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:19:32.966 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.966 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.966 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.966 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.966 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.966 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:19:32.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.966 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:32.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:19:32.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.900432869, 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.970 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:19:32.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.974 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:78833 3.a 3:5235bc0b:::rbd_data.11853dd604c.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.618163052, 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.35280560599999999, 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78833 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269309+0000", 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269322+0000", 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3359e-05 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269352+0000", 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0320000000000001e-05 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269367+0000", 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.526e-05 2024-09-13T09:19:32.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269434+0000", 2024-09-13T09:19:32.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.648e-05 2024-09-13T09:19:32.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:32.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269446+0000", 2024-09-13T09:19:32.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2286999999999999e-05 2024-09-13T09:19:32.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.606711+0000", 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.33726426399999998 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.606879+0000", 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016884100000000001 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.609371+0000", 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024920110000000001 2024-09-13T09:19:32.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622034+0000", 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012662199000000001 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622042+0000", 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9570000000000002e-06 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622083+0000", 2024-09-13T09:19:32.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.197e-05 2024-09-13T09:19:32.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622117+0000", 2024-09-13T09:19:32.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3013999999999999e-05 2024-09-13T09:19:32.981 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.981 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.981 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.981 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:80090 3.a 3:517b773f:::rbd_object_map.118553778f1e:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T09:19:32.981 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.161789756000001, 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99537752499999999, 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80090 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:32.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725681+0000", 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725699+0000", 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7683999999999999e-05 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725746+0000", 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7196e-05 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725769+0000", 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3383e-05 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725956+0000", 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000187049 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725985+0000", 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8472000000000001e-05 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.718483+0000", 2024-09-13T09:19:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99249815200000002 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.718851+0000", 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036844299999999998 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.719201+0000", 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034928699999999999 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720843+0000", 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016425249999999999 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720882+0000", 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9045999999999999e-05 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720985+0000", 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103163 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.721062+0000", 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6402000000000005e-05 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:81122 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.986949342999999, 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.920409343, 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:32.988 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 81122 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:32.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900523+0000", 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900536+0000", 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3200000000000001e-05 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900565+0000", 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9703e-05 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900588+0000", 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2096e-05 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900698+0000", 2024-09-13T09:19:32.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110029 2024-09-13T09:19:32.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:32.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900716+0000", 2024-09-13T09:19:32.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8070000000000001e-05 2024-09-13T09:19:32.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:17.766722+0000", 2024-09-13T09:19:32.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.86600628099999999 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:17.766770+0000", 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8486000000000003e-05 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.817359+0000", 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.050588836 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.817852+0000", 2024-09-13T09:19:32.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00049274599999999996 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.818512+0000", 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00066035099999999999 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820216+0000", 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017038680000000001 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820601+0000", 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038506599999999999 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820746+0000", 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014458900000000001 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820934+0000", 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188102 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.996 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.996 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:32.996 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:32.996 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:32.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:19:32.997 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:32.998 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:32.998 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:32.992+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:32.999 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:19:33.016 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:33.036 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:33.037 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:33.053 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:33.054 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:33.072 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:33.072 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:33.092 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:33.097 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:33.092+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:33.112 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:33.134 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:33.134 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:33.136 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:33.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:19:33.199 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:33.197+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:33.273 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:33.274 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:33.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:19:33.299 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:33.292+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:33.401 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:33.397+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:33.457 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:33.457 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:33.473 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:19:33.501 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:33.496+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:33.602 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:33.601+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:33.637 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:33.637 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:33.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:19:33.703 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:33.700+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:33.805 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:33.800+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:33.821 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:33.822 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:33.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:19:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:19:33.890 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.6 primary-affinity to 0.925232 (8606352) 2024-09-13T09:19:33.905 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:33.905+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:33.979 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:33.979 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:33.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:19:34.007 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:34.000+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:34.108 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:34.105+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:34.169 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:34.169 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:34.183 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:19:34.208 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:34.204+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:34.310 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:34.305+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:34.354 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:34.355 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:34.369 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:19:34.414 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:34.413+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:34.510 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:34.509+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:34.541 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:34.542 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:34.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:19:34.612 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:34.608+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:34.692 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:34.693 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:34.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:19:34.712 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:34.709+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:34.813 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:34.808+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:34.880 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:34.880 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:34.898 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:19:34.914 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:34.913+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:35.015 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:35.008+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:35.073 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:35.073 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:35.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:19:35.128 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:35.124+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:35.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:19:35.217 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:35.213+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:35.250 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:35.250 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:35.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:19:35.314 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:35.314 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:35.317 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:35.312+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:35.331 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:35.331 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:35.348 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:35.349 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:35.364 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:35.365 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:35.385 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:35.385 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:35.397 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:35.397 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:35.397 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:35.397 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:35.397 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:35.405 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:35.406 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:35.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:19:35.418 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:35.417+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:35.427 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:35.449 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:35.449 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:35.519 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:35.517+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:35.561 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:19:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 89.485110000999995, 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:19:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:19:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56102 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5439488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.142268944000001, 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060495140000000003, 2024-09-13T09:19:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56102 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416845+0000", 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416853+0000", 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:19:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416862+0000", 2024-09-13T09:19:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1959999999999997e-06 2024-09-13T09:19:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416868+0000", 2024-09-13T09:19:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9719999999999997e-06 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477088+0000", 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060219898000000001 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477154+0000", 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5605000000000006e-05 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477341+0000", 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018687599999999999 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56103 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5537792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.142240880000003, 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060576972999999999, 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56103 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416873+0000", 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416876+0000", 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.993e-06 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416883+0000", 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.477e-06 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416888+0000", 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2580000000000003e-06 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477357+0000", 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060469055000000001 2024-09-13T09:19:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477396+0000", 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9402e-05 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477451+0000", 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4262000000000003e-05 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56104 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5636096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.142222388, 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060778881999999999, 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56104 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416894+0000", 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4049999999999998e-06 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416900+0000", 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.959e-06 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416905+0000", 2024-09-13T09:19:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477465+0000", 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060559460000000002 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477521+0000", 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5813000000000002e-05 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477671+0000", 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150366 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56105 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5734400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.142178176999998, 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060898832, 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56105 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416939+0000", 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3550000000000001e-06 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416947+0000", 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0949999999999996e-06 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416953+0000", 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6790000000000002e-06 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477708+0000", 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060755454 2024-09-13T09:19:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477760+0000", 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1589999999999999e-05 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477835+0000", 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5394000000000004e-05 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56106 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5832704~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.14215686, 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061142999000000003, 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56106 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416957+0000", 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416960+0000", 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6979999999999998e-06 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416967+0000", 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6919999999999996e-06 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416973+0000", 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5679999999999996e-06 2024-09-13T09:19:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477867+0000", 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060894082000000002 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477924+0000", 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7015999999999998e-05 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478101+0000", 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017648300000000001 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56107 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5931008~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.141828246000003, 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060956639999999999, 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56107 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416977+0000", 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996905 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417291+0000", 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031392799999999999 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417311+0000", 2024-09-13T09:19:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0343000000000002e-05 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417329+0000", 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7612e-05 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478123+0000", 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060794421000000001 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478174+0000", 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0614e-05 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478243+0000", 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9226000000000004e-05 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56108 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 6029312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.141768761999998, 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061163618000000003, 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56108 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417345+0000", 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417349+0000", 2024-09-13T09:19:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7419999999999999e-06 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417367+0000", 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7714999999999999e-05 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417382+0000", 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5396999999999998e-05 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478270+0000", 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060888022999999999 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478322+0000", 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2234999999999997e-05 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478509+0000", 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187109 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56121 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7307264~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.137785268999998, 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060455939, 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56121 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.600 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:35.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421332+0000", 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421358+0000", 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6427999999999998e-05 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421374+0000", 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6137000000000002e-05 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481604+0000", 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060229528999999997 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481680+0000", 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6089999999999998e-05 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481785+0000", 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105259 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56122 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7405568~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.137711766000002, 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060604366999999999, 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.604 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56122 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421402+0000", 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421406+0000", 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.337e-06 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421415+0000", 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421434+0000", 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8617000000000001e-05 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481827+0000", 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060393883000000002 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481910+0000", 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2657999999999994e-05 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482007+0000", 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7057000000000003e-05 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.608 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56123 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7503872~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.137675227999999, 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060838125999999999, 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56123 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421450+0000", 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1544e-05 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421463+0000", 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421468+0000", 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6250000000000004e-06 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482066+0000", 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060597713999999997 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482163+0000", 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7112000000000001e-05 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482277+0000", 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114274 2024-09-13T09:19:35.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56124 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7602176~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.137641273, 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061054929000000001, 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56124 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421476+0000", 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2059999999999999e-06 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421484+0000", 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7889999999999995e-06 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421489+0000", 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482321+0000", 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060831477000000002 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482403+0000", 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2776999999999997e-05 2024-09-13T09:19:35.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482528+0000", 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012491600000000001 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56125 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7700480~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.137620976000001, 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061273721000000003, 2024-09-13T09:19:35.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56125 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421493+0000", 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421496+0000", 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.666e-06 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421503+0000", 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421508+0000", 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0919999999999997e-06 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482575+0000", 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061066339999999997 2024-09-13T09:19:35.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482659+0000", 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4158000000000003e-05 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482767+0000", 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010849 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56126 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7798784~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.137601609999997, 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061489349999999998, 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56126 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421531+0000", 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8218000000000001e-05 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421538+0000", 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.345e-06 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421544+0000", 2024-09-13T09:19:35.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482813+0000", 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061268959999999997 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482900+0000", 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6858000000000001e-05 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483002+0000", 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102599 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56127 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7897088~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.137565868999999, 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061729777, 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56127 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421548+0000", 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421551+0000", 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421559+0000", 2024-09-13T09:19:35.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4499999999999998e-06 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421564+0000", 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483067+0000", 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061503073999999998 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483153+0000", 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483278+0000", 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012522499999999999 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56319 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1613824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:35.630 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.131808122999999, 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060599663999999998, 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56319 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427309+0000", 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.672e-06 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427316+0000", 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6850000000000001e-06 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427322+0000", 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487756+0000", 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060434817000000002 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487825+0000", 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8553000000000006e-05 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487906+0000", 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1067000000000002e-05 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.634 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56320 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1712128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.131788309000001, 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060803930999999999, 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56320 2024-09-13T09:19:35.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427328+0000", 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427379+0000", 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0343999999999998e-05 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427385+0000", 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3969999999999999e-06 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487941+0000", 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060555805999999997 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488034+0000", 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:19:35.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488130+0000", 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6348999999999995e-05 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56321 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1810432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.131723372000003, 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060999937999999997, 2024-09-13T09:19:35.639 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56321 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427394+0000", 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.721e-06 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427403+0000", 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7110000000000001e-06 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427409+0000", 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9710000000000001e-06 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488183+0000", 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060774003 2024-09-13T09:19:35.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488291+0000", 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010786899999999999 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488391+0000", 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100142 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56322 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1908736~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.131377256999997, 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060864109999999999, 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56322 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427433+0000", 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996958 2024-09-13T09:19:35.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427742+0000", 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000309084 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427754+0000", 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1354e-05 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427761+0000", 2024-09-13T09:19:35.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8059999999999995e-06 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488429+0000", 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060667139000000002 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488496+0000", 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7595000000000006e-05 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488601+0000", 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105257 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56323 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 2007040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.130771562, 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060546618000000003, 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:35.648 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56323 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427767+0000", 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994245 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428348+0000", 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00058032600000000004 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428360+0000", 2024-09-13T09:19:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2103000000000001e-05 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428367+0000", 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1949999999999997e-06 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488650+0000", 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060282626999999998 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:35.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488745+0000", 2024-09-13T09:19:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5365999999999995e-05 2024-09-13T09:19:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488890+0000", 2024-09-13T09:19:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000144506 2024-09-13T09:19:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:35.652 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:35.652 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:35.654 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:19:35.654 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:35.648+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:35.755 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:35.753+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:35.826 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:35.826 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:35.826 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:35.826 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:35.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:19:35.892 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:35.888+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:35.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:19:35.968 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:35.964+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:35.981 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:36.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:19:36.092 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:36.088+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:36.160 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:36.157+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:36.172 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33548 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.178879792000004, 2024-09-13T09:19:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025760904000000001, 2024-09-13T09:19:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33548 2024-09-13T09:19:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:36.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990124+0000", 2024-09-13T09:19:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:19:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990130+0000", 2024-09-13T09:19:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:19:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990135+0000", 2024-09-13T09:19:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:19:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007826+0000", 2024-09-13T09:19:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017691018999999999 2024-09-13T09:19:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007896+0000", 2024-09-13T09:19:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0059000000000001e-05 2024-09-13T09:19:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.015883+0000", 2024-09-13T09:19:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079875269999999995 2024-09-13T09:19:36.178 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33751 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1593344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175460138000005, 2024-09-13T09:19:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023779101, 2024-09-13T09:19:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.179 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33751 2024-09-13T09:19:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:36.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993548+0000", 2024-09-13T09:19:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.776e-06 2024-09-13T09:19:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993555+0000", 2024-09-13T09:19:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:19:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993560+0000", 2024-09-13T09:19:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:19:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017188+0000", 2024-09-13T09:19:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023627984000000001 2024-09-13T09:19:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017271+0000", 2024-09-13T09:19:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2798000000000002e-05 2024-09-13T09:19:36.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017321+0000", 2024-09-13T09:19:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0557999999999999e-05 2024-09-13T09:19:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33752 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175438017999994, 2024-09-13T09:19:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023925173000000001, 2024-09-13T09:19:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.185 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33752 2024-09-13T09:19:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:36.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993566+0000", 2024-09-13T09:19:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:19:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993573+0000", 2024-09-13T09:19:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.759e-06 2024-09-13T09:19:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993578+0000", 2024-09-13T09:19:36.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8339999999999998e-06 2024-09-13T09:19:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017349+0000", 2024-09-13T09:19:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023770959000000001 2024-09-13T09:19:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017423+0000", 2024-09-13T09:19:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3652000000000006e-05 2024-09-13T09:19:36.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017489+0000", 2024-09-13T09:19:36.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6748999999999994e-05 2024-09-13T09:19:36.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.191 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33753 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1650688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.191 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:36.191 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175419536000007, 2024-09-13T09:19:36.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024066349000000001, 2024-09-13T09:19:36.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33753 2024-09-13T09:19:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993582+0000", 2024-09-13T09:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993585+0000", 2024-09-13T09:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993591+0000", 2024-09-13T09:19:36.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:19:36.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993595+0000", 2024-09-13T09:19:36.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:19:36.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017515+0000", 2024-09-13T09:19:36.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023920117000000001 2024-09-13T09:19:36.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017583+0000", 2024-09-13T09:19:36.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8485999999999994e-05 2024-09-13T09:19:36.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017649+0000", 2024-09-13T09:19:36.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:19:36.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.196 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.196 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33754 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.196 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:36.196 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175403242000002, 2024-09-13T09:19:36.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02420106, 2024-09-13T09:19:36.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.197 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.197 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.197 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.197 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.197 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33754 2024-09-13T09:19:36.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.197 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:36.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:36.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:36.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993601+0000", 2024-09-13T09:19:36.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:19:36.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993607+0000", 2024-09-13T09:19:36.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9669999999999998e-06 2024-09-13T09:19:36.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993611+0000", 2024-09-13T09:19:36.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:19:36.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017669+0000", 2024-09-13T09:19:36.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024057683999999999 2024-09-13T09:19:36.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017742+0000", 2024-09-13T09:19:36.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3397000000000002e-05 2024-09-13T09:19:36.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017800+0000", 2024-09-13T09:19:36.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7711999999999999e-05 2024-09-13T09:19:36.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.203 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.203 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.203 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33755 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1708032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.203 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:36.203 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175386822999997, 2024-09-13T09:19:36.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024346923999999999, 2024-09-13T09:19:36.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.204 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33755 2024-09-13T09:19:36.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.204 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:36.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:36.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:36.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993617+0000", 2024-09-13T09:19:36.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:19:36.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993624+0000", 2024-09-13T09:19:36.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0469999999999999e-06 2024-09-13T09:19:36.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993628+0000", 2024-09-13T09:19:36.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:19:36.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017830+0000", 2024-09-13T09:19:36.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024201879999999999 2024-09-13T09:19:36.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017897+0000", 2024-09-13T09:19:36.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7096999999999998e-05 2024-09-13T09:19:36.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017962+0000", 2024-09-13T09:19:36.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5455000000000003e-05 2024-09-13T09:19:36.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.211 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33756 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.211 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:36.211 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175370096999998, 2024-09-13T09:19:36.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024485512000000001, 2024-09-13T09:19:36.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.211 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.211 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33756 2024-09-13T09:19:36.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.212 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:36.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:36.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:36.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993635+0000", 2024-09-13T09:19:36.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:19:36.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993641+0000", 2024-09-13T09:19:36.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:19:36.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993645+0000", 2024-09-13T09:19:36.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2520000000000001e-06 2024-09-13T09:19:36.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017979+0000", 2024-09-13T09:19:36.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024333950999999999 2024-09-13T09:19:36.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018045+0000", 2024-09-13T09:19:36.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5843e-05 2024-09-13T09:19:36.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018118+0000", 2024-09-13T09:19:36.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2417999999999997e-05 2024-09-13T09:19:36.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.216 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.216 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33757 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1765376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.216 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:36.217 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175352699000001, 2024-09-13T09:19:36.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024605578999999999, 2024-09-13T09:19:36.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.217 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.217 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.217 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.217 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33757 2024-09-13T09:19:36.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.217 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:36.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:36.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993649+0000", 2024-09-13T09:19:36.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993651+0000", 2024-09-13T09:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.841e-06 2024-09-13T09:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993657+0000", 2024-09-13T09:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-13T09:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993662+0000", 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018139+0000", 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024476748 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018196+0000", 2024-09-13T09:19:36.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7188000000000002e-05 2024-09-13T09:19:36.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018255+0000", 2024-09-13T09:19:36.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9045999999999997e-05 2024-09-13T09:19:36.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.221 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.221 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33758 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.221 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:36.222 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175336066, 2024-09-13T09:19:36.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024736898, 2024-09-13T09:19:36.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.222 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.222 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33758 2024-09-13T09:19:36.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.222 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:36.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993668+0000", 2024-09-13T09:19:36.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:19:36.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993674+0000", 2024-09-13T09:19:36.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993679+0000", 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.515e-06 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018284+0000", 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024604816000000002 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018356+0000", 2024-09-13T09:19:36.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2782999999999998e-05 2024-09-13T09:19:36.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018403+0000", 2024-09-13T09:19:36.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6726e-05 2024-09-13T09:19:36.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.226 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.227 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33759 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1822720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.227 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:36.227 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175319459999997, 2024-09-13T09:19:36.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873635000000002, 2024-09-13T09:19:36.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.227 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33759 2024-09-13T09:19:36.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.227 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993682+0000", 2024-09-13T09:19:36.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993685+0000", 2024-09-13T09:19:36.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:19:36.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993691+0000", 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1840000000000004e-06 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993696+0000", 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018429+0000", 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024733330000000001 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018494+0000", 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5233000000000004e-05 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018556+0000", 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2125999999999994e-05 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33760 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175302604999999, 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025058980000000002, 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33760 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:36.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:36.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993699+0000", 2024-09-13T09:19:36.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993702+0000", 2024-09-13T09:19:36.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.632e-06 2024-09-13T09:19:36.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993708+0000", 2024-09-13T09:19:36.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2280000000000003e-06 2024-09-13T09:19:36.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993713+0000", 2024-09-13T09:19:36.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:19:36.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018586+0000", 2024-09-13T09:19:36.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873572 2024-09-13T09:19:36.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018645+0000", 2024-09-13T09:19:36.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9098000000000002e-05 2024-09-13T09:19:36.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018759+0000", 2024-09-13T09:19:36.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113224 2024-09-13T09:19:36.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.237 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.237 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33761 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1880064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.237 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175285908000006, 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025229076, 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33761 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:36.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993718+0000", 2024-09-13T09:19:36.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.092e-06 2024-09-13T09:19:36.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993724+0000", 2024-09-13T09:19:36.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0839999999999997e-06 2024-09-13T09:19:36.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993729+0000", 2024-09-13T09:19:36.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:19:36.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018790+0000", 2024-09-13T09:19:36.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025061039 2024-09-13T09:19:36.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018871+0000", 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0885999999999998e-05 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018945+0000", 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4773999999999999e-05 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33762 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:36.242 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175269434000001, 2024-09-13T09:19:36.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025559609, 2024-09-13T09:19:36.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33762 2024-09-13T09:19:36.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.243 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:36.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:36.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993732+0000", 2024-09-13T09:19:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993735+0000", 2024-09-13T09:19:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:19:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993740+0000", 2024-09-13T09:19:36.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993745+0000", 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5290000000000002e-06 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019043+0000", 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025298355000000002 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019134+0000", 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0781999999999999e-05 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019292+0000", 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015829 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33763 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.247 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:36.248 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175253493, 2024-09-13T09:19:36.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025732657999999999, 2024-09-13T09:19:36.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.248 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.248 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.248 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.248 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.248 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33763 2024-09-13T09:19:36.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.249 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:36.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:36.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993748+0000", 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993755+0000", 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993762+0000", 2024-09-13T09:19:36.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993767+0000", 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7230000000000004e-06 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019322+0000", 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025555564999999999 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019402+0000", 2024-09-13T09:19:36.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9433999999999999e-05 2024-09-13T09:19:36.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019481+0000", 2024-09-13T09:19:36.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9739000000000003e-05 2024-09-13T09:19:36.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.252 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.253 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33764 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.253 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:36.253 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175231119000003, 2024-09-13T09:19:36.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025908526000000001, 2024-09-13T09:19:36.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.253 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.253 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.253 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.253 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33764 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993773+0000", 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993779+0000", 2024-09-13T09:19:36.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9610000000000002e-06 2024-09-13T09:19:36.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993783+0000", 2024-09-13T09:19:36.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.459e-06 2024-09-13T09:19:36.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019513+0000", 2024-09-13T09:19:36.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025729399 2024-09-13T09:19:36.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019609+0000", 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6504999999999998e-05 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019680+0000", 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0393000000000002e-05 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33765 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1994752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175214883999999, 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026051009999999999, 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33765 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.258 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993789+0000", 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993795+0000", 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.088e-06 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993799+0000", 2024-09-13T09:19:36.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:19:36.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019704+0000", 2024-09-13T09:19:36.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025904311999999999 2024-09-13T09:19:36.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019766+0000", 2024-09-13T09:19:36.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2829000000000004e-05 2024-09-13T09:19:36.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019838+0000", 2024-09-13T09:19:36.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1935999999999997e-05 2024-09-13T09:19:36.262 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.262 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.262 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.262 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33766 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.262 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:36.263 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175198828999996, 2024-09-13T09:19:36.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026238971999999999, 2024-09-13T09:19:36.263 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.263 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.263 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.263 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.263 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.263 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33766 2024-09-13T09:19:36.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.263 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:36.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:36.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:36.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993805+0000", 2024-09-13T09:19:36.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:19:36.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993811+0000", 2024-09-13T09:19:36.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9000000000000003e-06 2024-09-13T09:19:36.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993816+0000", 2024-09-13T09:19:36.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.527e-06 2024-09-13T09:19:36.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019877+0000", 2024-09-13T09:19:36.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026061337 2024-09-13T09:19:36.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019960+0000", 2024-09-13T09:19:36.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3245999999999995e-05 2024-09-13T09:19:36.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020042+0000", 2024-09-13T09:19:36.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2053999999999997e-05 2024-09-13T09:19:36.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.268 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.268 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33767 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.268 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:36.268 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175182418999995, 2024-09-13T09:19:36.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026407013999999999, 2024-09-13T09:19:36.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33767 2024-09-13T09:19:36.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.269 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:36.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:36.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993819+0000", 2024-09-13T09:19:36.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993822+0000", 2024-09-13T09:19:36.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:19:36.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993828+0000", 2024-09-13T09:19:36.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:19:36.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993832+0000", 2024-09-13T09:19:36.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.493e-06 2024-09-13T09:19:36.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020072+0000", 2024-09-13T09:19:36.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026239519999999999 2024-09-13T09:19:36.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020151+0000", 2024-09-13T09:19:36.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9278999999999998e-05 2024-09-13T09:19:36.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020227+0000", 2024-09-13T09:19:36.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5965999999999997e-05 2024-09-13T09:19:36.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.274 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33768 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.175163966, 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026564424, 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33768 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:36.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:36.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:36.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993840+0000", 2024-09-13T09:19:36.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.996e-06 2024-09-13T09:19:36.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993846+0000", 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993858+0000", 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2214e-05 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020253+0000", 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026394926999999999 2024-09-13T09:19:36.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020329+0000", 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5193999999999999e-05 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020403+0000", 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4145000000000002e-05 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.280 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:78829 3.6 3:6418b130:::rbd_data.11853dd604c.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:36.280 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:36.280 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 36.952477772000002, 2024-09-13T09:19:36.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041141776999999997, 2024-09-13T09:19:36.280 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.280 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:36.280 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.280 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.280 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.280 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 78829 2024-09-13T09:19:36.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216522+0000", 2024-09-13T09:19:36.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:36.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216540+0000", 2024-09-13T09:19:36.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8280000000000001e-05 2024-09-13T09:19:36.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216577+0000", 2024-09-13T09:19:36.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7151999999999999e-05 2024-09-13T09:19:36.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216594+0000", 2024-09-13T09:19:36.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7113e-05 2024-09-13T09:19:36.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216696+0000", 2024-09-13T09:19:36.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010190400000000001 2024-09-13T09:19:36.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216833+0000", 2024-09-13T09:19:36.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136697 2024-09-13T09:19:36.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.224657+0000", 2024-09-13T09:19:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078241000000000005 2024-09-13T09:19:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257365+0000", 2024-09-13T09:19:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032707984000000002 2024-09-13T09:19:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:36.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257404+0000", 2024-09-13T09:19:36.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8979000000000001e-05 2024-09-13T09:19:36.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:36.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257549+0000", 2024-09-13T09:19:36.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000145737 2024-09-13T09:19:36.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257666+0000", 2024-09-13T09:19:36.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116754 2024-09-13T09:19:36.287 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.287 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.287 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.287 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.287 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.287 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:36.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:19:36.289 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:36.285+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:36.390 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:36.384+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:36.464 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:36.465 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:36.480 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:19:36.491 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:36.489+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:36.592 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:36.589+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:36.652 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:36.653 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:36.653 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:36.653 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:36.653 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:36.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:19:36.693 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:36.689+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:36.792 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:36.792 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:36.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:36.793 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:36.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.793 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:19:36.793 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:36.793 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 109.254245931, 2024-09-13T09:19:36.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:19:36.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.793 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:19:36.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:19:36.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:19:36.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:19:36.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:19:36.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:19:36.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:19:36.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:19:36.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:19:36.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:36.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:19:36.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:19:36.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:36.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:19:36.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:19:36.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:36.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:19:36.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:19:36.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:36.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:19:36.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:19:36.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:19:36.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:19:36.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.800 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.800 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.800 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:36.800 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884678945999994, 2024-09-13T09:19:36.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:19:36.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.801 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.801 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.801 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.801 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.801 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:19:36.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.801 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:36.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:36.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:19:36.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:19:36.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:19:36.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:19:36.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:19:36.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:19:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:19:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:19:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:19:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:19:36.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:19:36.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.805 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.805 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:36.805 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884661625000007, 2024-09-13T09:19:36.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:19:36.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:19:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:19:36.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:19:36.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:19:36.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:19:36.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:19:36.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:19:36.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:19:36.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:19:36.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:19:36.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:19:36.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.810 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.810 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:36.810 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884644683999994, 2024-09-13T09:19:36.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:19:36.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.811 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.811 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.811 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.811 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:19:36.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.811 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:36.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:36.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:19:36.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884627456000004, 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.817 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.817 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:19:36.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.817 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:36.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:36.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:36.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:19:36.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:19:36.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:19:36.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:19:36.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:19:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:19:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:19:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:36.821 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884591035, 2024-09-13T09:19:36.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:19:36.822 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.822 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.822 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:19:36.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.822 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:36.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:36.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:36.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:19:36.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:19:36.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:19:36.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884537573000003, 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.827 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.828 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:19:36.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.828 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:36.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:19:36.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884515553, 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.832 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:19:36.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884490917999997, 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.837 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:19:36.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.837 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:36.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:19:36.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:19:36.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:19:36.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:19:36.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:19:36.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:19:36.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:19:36.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:19:36.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:19:36.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:19:36.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.841 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.841 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:36.841 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884472395000003, 2024-09-13T09:19:36.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:19:36.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:36.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:36.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:19:36.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:19:36.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:19:36.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:19:36.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:19:36.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:19:36.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:19:36.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:19:36.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:19:36.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:19:36.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.846 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.846 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:36.846 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884454524000006, 2024-09-13T09:19:36.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:19:36.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:36.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:19:36.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:19:36.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:19:36.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884437194, 2024-09-13T09:19:36.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:36.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:36.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:19:36.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:19:36.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:19:36.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:19:36.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:19:36.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:19:36.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:19:36.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:19:36.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:19:36.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:19:36.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884419723999997, 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.856 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:19:36.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.856 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:36.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:36.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:19:36.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884402858000001, 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:19:36.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:19:36.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:19:36.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:19:36.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:19:36.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.864 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.864 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.864 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:36.864 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884385070999997, 2024-09-13T09:19:36.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:19:36.864 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.864 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.864 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.864 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.865 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.865 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:19:36.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.865 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:36.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:36.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:36.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:19:36.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:19:36.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:19:36.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:19:36.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:19:36.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:19:36.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:19:36.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:19:36.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:19:36.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:19:36.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.869 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.869 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:36.869 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.884340761000004, 2024-09-13T09:19:36.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:19:36.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.870 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:36.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.870 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:19:36.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.870 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:36.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:36.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:19:36.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:19:36.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:19:36.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:19:36.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:19:36.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:19:36.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:36.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:19:36.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:19:36.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:19:36.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:19:36.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.874 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.874 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.651210273000004, 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:36.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:36.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:36.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:19:36.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:19:36.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:19:36.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.879 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:36.880 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.65119344, 2024-09-13T09:19:36.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:19:36.880 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.880 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.880 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:19:36.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.881 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:36.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:19:36.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:19:36.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:19:36.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:19:36.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:19:36.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:19:36.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:19:36.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:19:36.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:19:36.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:19:36.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:19:36.885 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.885 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.885 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.885 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:36.885 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:36.885 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.651177082000004, 2024-09-13T09:19:36.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:19:36.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.885 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:36.885 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.886 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.886 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.886 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:19:36.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.886 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:36.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:19:36.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:19:36.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:19:36.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:19:36.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:19:36.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:19:36.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:19:36.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:19:36.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:66592 3.7 3:e0817996:::rbd_data.1185b1e141e2.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 57.146817351999999, 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.051168777999999998, 2024-09-13T09:19:36.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 66592 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643787+0000", 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643799+0000", 2024-09-13T09:19:36.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2140000000000001e-05 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643827+0000", 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8121999999999999e-05 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643841+0000", 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3969e-05 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643985+0000", 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014404199999999999 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.644371+0000", 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038588700000000001 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.654317+0000", 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099452050000000004 2024-09-13T09:19:36.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:36.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694641+0000", 2024-09-13T09:19:36.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.040324802999999999 2024-09-13T09:19:36.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:36.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694682+0000", 2024-09-13T09:19:36.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0451999999999998e-05 2024-09-13T09:19:36.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694828+0000", 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014580499999999999 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694958+0000", 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013029599999999999 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:36.896 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:36.898 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:19:36.898 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:36.897+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:36.925 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:19:36.999 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:36.992+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:37.040 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:37.040 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:37.041 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:37.041 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:37.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:19:37.101 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:37.101+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:37.181 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:37.181 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:37.181 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:37.181 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:37.181 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:37.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:19:37.238 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:37.237+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:37.302 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:37.296+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:37.369 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.379290342, 2024-09-13T09:19:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:19:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:19:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:19:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:19:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:19:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:19:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:19:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:19:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:19:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:19:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:19:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.37924160599999, 2024-09-13T09:19:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:19:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:19:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:19:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:19:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:19:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:19:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:19:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:19:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:19:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:19:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:19:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:19:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:19:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:19:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:19:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.379173937, 2024-09-13T09:19:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:19:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:19:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:19:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:19:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:19:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:19:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:19:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:19:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:19:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:19:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:19:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:19:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:19:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:19:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:19:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.379033728, 2024-09-13T09:19:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:19:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:19:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:19:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:19:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:19:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:19:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:19:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:19:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:19:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:19:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:19:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.378942862, 2024-09-13T09:19:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:19:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:19:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:19:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:19:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:19:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:19:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:19:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:19:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:19:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:19:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:19:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:19:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:19:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:19:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:19:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.378868002, 2024-09-13T09:19:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:19:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:19:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:19:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:19:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:19:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:19:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:19:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:19:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.378796826, 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:19:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:19:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:19:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:19:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:19:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:19:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:19:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:19:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:19:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:19:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:19:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:19:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:19:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.378747773, 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:19:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:19:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:19:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:19:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:19:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:19:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:19:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:19:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:19:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:19:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:19:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:19:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:19:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.37868603699999, 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:19:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:19:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:19:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:19:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:19:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:19:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:19:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:19:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:19:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:19:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:19:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.378636367, 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:19:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:19:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:19:37.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:19:37.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:19:37.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:19:37.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:19:37.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:19:37.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:19:37.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:19:37.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:19:37.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.378587848, 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.427 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.427 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.427 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:19:37.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.427 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:37.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:37.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:19:37.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:19:37.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:19:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:19:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:19:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:19:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:19:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.378539452, 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:19:37.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.432 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:37.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:19:37.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:19:37.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:19:37.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:19:37.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:19:37.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:19:37.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.378492165, 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.437 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:19:37.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.437 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:37.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:19:37.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:19:37.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:19:37.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:19:37.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:19:37.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:19:37.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.440 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.441 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.441 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.441 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:37.441 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.378408763, 2024-09-13T09:19:37.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:19:37.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:19:37.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.442 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:37.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:19:37.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:19:37.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:19:37.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:19:37.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:19:37.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:19:37.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:19:37.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.446 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.446 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.446 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:37.446 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.378335037, 2024-09-13T09:19:37.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:19:37.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.446 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:37.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:19:37.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:19:37.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:19:37.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:19:37.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:19:37.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:19:37.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:19:37.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:19:37.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.37826691399999, 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:19:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:19:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:19:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:19:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:19:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:19:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 113.378216447, 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.456 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:19:37.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:37.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:19:37.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:19:37.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:19:37.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:19:37.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:19:37.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:19:37.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.460 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:78833 3.a 3:5235bc0b:::rbd_data.11853dd604c.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.09594663, 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.35280560599999999, 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78833 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269309+0000", 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269322+0000", 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3359e-05 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269352+0000", 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0320000000000001e-05 2024-09-13T09:19:37.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269367+0000", 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.526e-05 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269434+0000", 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.648e-05 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269446+0000", 2024-09-13T09:19:37.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2286999999999999e-05 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.606711+0000", 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.33726426399999998 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.606879+0000", 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016884100000000001 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.609371+0000", 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024920110000000001 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622034+0000", 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012662199000000001 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622042+0000", 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9570000000000002e-06 2024-09-13T09:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622083+0000", 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.197e-05 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622117+0000", 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3013999999999999e-05 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.468 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:80090 3.a 3:517b773f:::rbd_object_map.118553778f1e:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T09:19:37.468 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:37.468 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.639573334000001, 2024-09-13T09:19:37.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99537752499999999, 2024-09-13T09:19:37.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:37.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80090 2024-09-13T09:19:37.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.469 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:37.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:37.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725681+0000", 2024-09-13T09:19:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-13T09:19:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725699+0000", 2024-09-13T09:19:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7683999999999999e-05 2024-09-13T09:19:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725746+0000", 2024-09-13T09:19:37.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7196e-05 2024-09-13T09:19:37.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725769+0000", 2024-09-13T09:19:37.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3383e-05 2024-09-13T09:19:37.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725956+0000", 2024-09-13T09:19:37.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000187049 2024-09-13T09:19:37.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:37.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725985+0000", 2024-09-13T09:19:37.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8472000000000001e-05 2024-09-13T09:19:37.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.718483+0000", 2024-09-13T09:19:37.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99249815200000002 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.718851+0000", 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036844299999999998 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.719201+0000", 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034928699999999999 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:37.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720843+0000", 2024-09-13T09:19:37.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016425249999999999 2024-09-13T09:19:37.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:37.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720882+0000", 2024-09-13T09:19:37.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9045999999999999e-05 2024-09-13T09:19:37.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720985+0000", 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103163 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.721062+0000", 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6402000000000005e-05 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:81122 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:19:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.464732921, 2024-09-13T09:19:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.920409343, 2024-09-13T09:19:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 81122 2024-09-13T09:19:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:37.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900523+0000", 2024-09-13T09:19:37.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:37.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:37.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900536+0000", 2024-09-13T09:19:37.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3200000000000001e-05 2024-09-13T09:19:37.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900565+0000", 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9703e-05 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900588+0000", 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2096e-05 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900698+0000", 2024-09-13T09:19:37.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110029 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900716+0000", 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8070000000000001e-05 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:17.766722+0000", 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.86600628099999999 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:17.766770+0000", 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8486000000000003e-05 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.817359+0000", 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.050588836 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.817852+0000", 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00049274599999999996 2024-09-13T09:19:37.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.818512+0000", 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00066035099999999999 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820216+0000", 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017038680000000001 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820601+0000", 2024-09-13T09:19:37.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038506599999999999 2024-09-13T09:19:37.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:37.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820746+0000", 2024-09-13T09:19:37.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014458900000000001 2024-09-13T09:19:37.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:37.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:37.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:37.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820934+0000", 2024-09-13T09:19:37.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188102 2024-09-13T09:19:37.484 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.484 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.484 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.484 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:37.484 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:37.484 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:37.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:19:37.486 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:37.484+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:37.488 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:19:37.586 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:37.585+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:37.620 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:37.620 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:37.620 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:37.620 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:37.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:19:37.645 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:37.661 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:37.661 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:37.677 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:37.677 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:37.687 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:37.680+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:37.693 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:37.696 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:37.711 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:37.729 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:37.729 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:37.747 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:37.747 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:37.766 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:37.806 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:37.807 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:37.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:19:37.828 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:37.824+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:37.888 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:37.885+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:37.955 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:37.955 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:37.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:19:37.989 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:37.984+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:38.090 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:38.089+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:38.150 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:38.150 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:38.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:19:38.191 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:38.184+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:38.292 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:38.288+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:38.342 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:38.343 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:38.357 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:19:38.393 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:38.392+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:38.481 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:38.481 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:38.494 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:38.493+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:38.495 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:19:38.595 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:38.593+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:38.658 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:38.658 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:38.672 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:19:38.696 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:38.692+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:38.797 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:38.796+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:38.841 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:38.841 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:38.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:19:38.897 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:38.892+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:38.921 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:19:38.921 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:19:38.921 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T09:19:38.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:19:38.950 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:19:38.989 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:38.990 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:38.998 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:38.997+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:39.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:19:39.099 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:39.097+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:39.160 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:39.160 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:39.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:19:39.220 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:39.216+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:39.268 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:19:39.268 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:39.268 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 136, 2024-09-13T09:19:39.268 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "f0889cb1-03d6-4b81-88c9-6141846c6033", 2024-09-13T09:19:39.268 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-13T09:16:56.964982+0000", 2024-09-13T09:19:39.268 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-13T09:19:38.925532+0000", 2024-09-13T09:19:39.269 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-13T09:17:08.368378+0000", 2024-09-13T09:19:39.269 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-13T09:19:16.749126+0000", 2024-09-13T09:19:39.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:19:39.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-09-13T09:19:39.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-13T09:19:39.269 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-13T09:19:39.269 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-13T09:19:39.269 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-13T09:19:39.269 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-13T09:19:39.269 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:39.269 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-09-13T09:19:39.269 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:19:39.270 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:19:39.270 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:19:39.270 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-13T09:19:39.270 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-13T09:19:39.270 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-13T09:19:39.270 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:19:39.270 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-09-13T09:19:39.270 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-13T09:19:39.270 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-13T09:19:39.270 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-13T09:19:39.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.270 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-13T09:17:10.140920+0000", 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:19:39.271 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-13T09:19:39.272 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:19:39.272 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-13T09:19:39.272 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-13T09:19:39.272 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-13T09:19:39.272 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-13T09:19:39.272 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-13T09:19:39.272 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-13T09:19:39.272 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-13T09:19:39.272 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-13T09:19:39.272 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-13T09:19:39.272 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "17", 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-13T09:19:39.273 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:19:39.274 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-13T09:19:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-13T09:19:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-13T09:19:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-13T09:19:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-13T09:19:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-13T09:19:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-13T09:19:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-13T09:19:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:19:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:19:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:19:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:19:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-13T09:19:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-13T09:19:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-13T09:19:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-13T09:19:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-13T09:19:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-13T09:19:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-13T09:19:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-13T09:19:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-13T09:19:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-13T09:19:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 6.9000000953674316, 2024-09-13T09:19:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 6.9000000953674316, 2024-09-13T09:19:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.28999999165534973, 2024-09-13T09:19:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-13T09:19:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-13T09:19:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.86565208435058594, 2024-09-13T09:19:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-13T09:19:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-13T09:19:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-13T09:17:15.775544+0000", 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-13T09:19:39.279 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-13T09:19:39.280 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-13T09:19:39.280 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-13T09:19:39.280 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-13T09:19:39.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.280 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "21", 2024-09-13T09:19:39.280 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-13T09:19:39.280 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:19:39.280 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:19:39.280 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-13T09:19:39.280 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:19:39.280 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-13T09:19:39.280 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 21, 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:19:39.281 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:19:39.282 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:19:39.282 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-13T09:19:39.282 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-13T09:19:39.282 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-13T09:19:39.282 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-13T09:19:39.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-13T09:19:39.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.282 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-13T09:19:39.282 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-13T09:19:39.282 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-13T09:19:39.282 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:19:39.282 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-09-13T09:19:39.283 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-09-13T09:19:39.284 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-09-13T09:19:39.284 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-09-13T09:19:39.284 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-09-13T09:19:39.284 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.86565208435058594, 2024-09-13T09:19:39.284 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-13T09:19:39.284 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-13T09:19:39.284 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.284 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-13T09:19:39.284 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-13T09:19:39.284 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-13T09:17:25.804931+0000", 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:19:39.285 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.286 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "136", 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 70, 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 136, 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-13T09:19:39.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-13T09:19:39.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-13T09:19:39.289 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3200000524520874, 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3200000524520874, 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1.1399999856948853, 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.86565208435058594, 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-13T09:19:39.290 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "786bdfc4-cc25-4167-aa5b-3737cfeb4ac1", 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:39.291 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 116, 2024-09-13T09:19:39.292 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:39.292 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:39.292 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:39.292 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.292 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-13T09:19:39.292 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:39.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.292 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:39.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.296 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.296 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-13T09:19:39.296 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:39.296 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.296 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.296 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/3549111487", 2024-09-13T09:19:39.296 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/3549111487", 2024-09-13T09:19:39.296 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/3549111487", 2024-09-13T09:19:39.296 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/3549111487", 2024-09-13T09:19:39.296 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "ca6189ab-90dd-4e9d-a85f-092186153b5a", 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:39.297 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 114, 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:39.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-13T09:19:39.301 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6810/1470371899", 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6815/1470371899", 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1470371899", 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6820/1470371899", 2024-09-13T09:19:39.302 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a", 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0, 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:39.303 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2024-09-13T09:19:39.304 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:39.304 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:39.304 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:39.304 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.304 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-13T09:19:39.304 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:39.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.304 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-13T09:19:39.305 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:39.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.305 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.305 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:39.305 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.305 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-13T09:19:39.305 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:39.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/2485123706", 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/2485123706", 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/2485123706", 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/2485123706", 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:39.308 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218", 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:39.309 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 125, 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-13T09:19:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-13T09:19:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-13T09:19:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-13T09:19:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-13T09:19:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-13T09:19:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/1009890408", 2024-09-13T09:19:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6814/1009890408", 2024-09-13T09:19:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6821/1009890408", 2024-09-13T09:19:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6818/1009890408", 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39f42e87-29dd-42cc-9ef6-e0074f1794e4", 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 117, 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-13T09:19:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-13T09:19:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-13T09:19:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-13T09:19:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6823/2723023689", 2024-09-13T09:19:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6825/2723023689", 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/2723023689", 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6828/2723023689", 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "cf8dd701-9543-4bdd-a6f6-0b56210f00c2", 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-13T09:19:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6809/2955019640", 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6812/2955019640", 2024-09-13T09:19:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6819/2955019640", 2024-09-13T09:19:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6815/2955019640", 2024-09-13T09:19:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-13T09:19:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "dd080b88-3292-4a34-8dbd-645683d8fe14", 2024-09-13T09:19:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.9252166748046875, 2024-09-13T09:19:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 131, 2024-09-13T09:19:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6813/916160151", 2024-09-13T09:19:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6818/916160151", 2024-09-13T09:19:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6829/916160151", 2024-09-13T09:19:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6822/916160151", 2024-09-13T09:19:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-13T09:19:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c6bca54b-e982-4d6a-9ae9-057ee3881a5b", 2024-09-13T09:19:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 101, 2024-09-13T09:19:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-13T09:19:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-13T09:19:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6801/3477167689", 2024-09-13T09:19:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6803/3477167689", 2024-09-13T09:19:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6807/3477167689", 2024-09-13T09:19:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6805/3477167689", 2024-09-13T09:19:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.247113+0000", 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.166872+0000", 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.451769+0000", 2024-09-13T09:19:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-13T09:19:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.458130+0000", 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.466129+0000", 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-13T09:19:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.444828+0000", 2024-09-13T09:19:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-13T09:19:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.425145+0000", 2024-09-13T09:19:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-13T09:19:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.484371+0000", 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-13T09:19:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-13T09:19:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-13T09:19:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-13T09:19:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-13T09:19:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-13T09:19:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-13T09:19:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-13T09:19:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-13T09:19:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-13T09:19:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 15, 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 9 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [ 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-13T09:19:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 15, 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-13T09:19:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:19:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:19:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-13T09:19:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:39.350 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:39.352 INFO:tasks.thrashosds.thrasher:j is {'epoch': 136, 'fsid': 'f0889cb1-03d6-4b81-88c9-6141846c6033', 'created': '2024-09-13T09:16:56.964982+0000', 'modified': '2024-09-13T09:19:38.925532+0000', 'last_up_change': '2024-09-13T09:17:08.368378+0000', 'last_in_change': '2024-09-13T09:19:16.749126+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:17:10.140920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8656520843505859, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:17:15.775544+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8656520843505859, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:17:25.804931+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 70, 'snap_epoch': 136, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8656520843505859, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.8700000047683716}}], 'osds': [{'osd': 0, 'uuid': '786bdfc4-cc25-4167-aa5b-3737cfeb4ac1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 3549111487}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 3549111487}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 3549111487}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 3549111487}]}, 'public_addr': '172.21.15.32:6801/3549111487', 'cluster_addr': '172.21.15.32:6803/3549111487', 'heartbeat_back_addr': '172.21.15.32:6807/3549111487', 'heartbeat_front_addr': '172.21.15.32:6805/3549111487', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'ca6189ab-90dd-4e9d-a85f-092186153b5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 114, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6810', 'nonce': 1470371899}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 1470371899}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1470371899}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6817', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6820', 'nonce': 1470371899}]}, 'public_addr': '172.21.15.32:6810/1470371899', 'cluster_addr': '172.21.15.32:6815/1470371899', 'heartbeat_back_addr': '172.21.15.32:6823/1470371899', 'heartbeat_front_addr': '172.21.15.32:6820/1470371899', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 2485123706}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 2485123706}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 2485123706}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 2485123706}]}, 'public_addr': '172.21.15.32:6825/2485123706', 'cluster_addr': '172.21.15.32:6827/2485123706', 'heartbeat_back_addr': '172.21.15.32:6831/2485123706', 'heartbeat_front_addr': '172.21.15.32:6829/2485123706', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 125, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6809', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 1009890408}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6813', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6814', 'nonce': 1009890408}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6819', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1009890408}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6818', 'nonce': 1009890408}]}, 'public_addr': '172.21.15.32:6811/1009890408', 'cluster_addr': '172.21.15.32:6814/1009890408', 'heartbeat_back_addr': '172.21.15.32:6821/1009890408', 'heartbeat_front_addr': '172.21.15.32:6818/1009890408', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39f42e87-29dd-42cc-9ef6-e0074f1794e4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6823', 'nonce': 2723023689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6824', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 2723023689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 2723023689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 2723023689}]}, 'public_addr': '172.21.15.46:6823/2723023689', 'cluster_addr': '172.21.15.46:6825/2723023689', 'heartbeat_back_addr': '172.21.15.46:6831/2723023689', 'heartbeat_front_addr': '172.21.15.46:6828/2723023689', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'cf8dd701-9543-4bdd-a6f6-0b56210f00c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6809', 'nonce': 2955019640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6811', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6812', 'nonce': 2955019640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 2955019640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6814', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 2955019640}]}, 'public_addr': '172.21.15.46:6809/2955019640', 'cluster_addr': '172.21.15.46:6812/2955019640', 'heartbeat_back_addr': '172.21.15.46:6819/2955019640', 'heartbeat_front_addr': '172.21.15.46:6815/2955019640', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'dd080b88-3292-4a34-8dbd-645683d8fe14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9252166748046875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 131, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6810', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6813', 'nonce': 916160151}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 916160151}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 916160151}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 916160151}]}, 'public_addr': '172.21.15.46:6813/916160151', 'cluster_addr': '172.21.15.46:6818/916160151', 'heartbeat_back_addr': '172.21.15.46:6829/916160151', 'heartbeat_front_addr': '172.21.15.46:6822/916160151', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c6bca54b-e982-4d6a-9ae9-057ee3881a5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6801', 'nonce': 3477167689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6802', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 3477167689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6806', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 3477167689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6805', 'nonce': 3477167689}]}, 'public_addr': '172.21.15.46:6801/3477167689', 'cluster_addr': '172.21.15.46:6803/3477167689', 'heartbeat_back_addr': '172.21.15.46:6807/3477167689', 'heartbeat_front_addr': '172.21.15.46:6805/3477167689', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.247113+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.166872+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.451769+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.458130+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.466129+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.444828+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.425145+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.484371+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 15, 'length': 9}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 15, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:19:39.352 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-13T09:19:39.353 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:39.353 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:39.353 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:19:39.353 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:39.344+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:39.452 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:39.448+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:39.520 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:39.520 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:39.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:19:39.553 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:39.549+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:39.653 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:39.653+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:39.706 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:39.706 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:39.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:19:39.754 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:39.753+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:39.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:19:39.849 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:39.849 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:39.855 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:39.848+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:39.864 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:19:39.927 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:39.927 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:39.952 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:19:39.952 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:39.952 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:39.955 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:39.952+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:39.969 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:39.969 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:39.990 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:39.990 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:40.013 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:40.033 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:40.034 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:40.048 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:40.048 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:40.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:40.048 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:22659 3.19 3:9884777b:::rbd_data.1185d6f8ec87.0000000000000018:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1561584~8075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.972483358000005, 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90417935800000004, 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22659 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074005+0000", 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074004+0000", 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074015+0000", 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0978999999999999e-05 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074028+0000", 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3456e-05 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074034+0000", 2024-09-13T09:19:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:19:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074107+0000", 2024-09-13T09:19:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3268000000000004e-05 2024-09-13T09:19:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-13T09:19:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.074121+0000", 2024-09-13T09:19:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4027e-05 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.975814+0000", 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.90169326800000005 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976044+0000", 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022951199999999999 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.976289+0000", 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024539199999999998 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.977715+0000", 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014258369999999999 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978075+0000", 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00035967300000000002 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978136+0000", 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1291000000000007e-05 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:06.978184+0000", 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8015e-05 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56102 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5439488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.629642301000004, 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060495140000000003, 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56102 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416846+0000", 2024-09-13T09:19:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416845+0000", 2024-09-13T09:19:40.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416853+0000", 2024-09-13T09:19:40.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:19:40.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416862+0000", 2024-09-13T09:19:40.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1959999999999997e-06 2024-09-13T09:19:40.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416868+0000", 2024-09-13T09:19:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9719999999999997e-06 2024-09-13T09:19:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477088+0000", 2024-09-13T09:19:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060219898000000001 2024-09-13T09:19:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477154+0000", 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5605000000000006e-05 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477341+0000", 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018687599999999999 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56103 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5537792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.629614236999998, 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060576972999999999, 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56103 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416874+0000", 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416873+0000", 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416876+0000", 2024-09-13T09:19:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.993e-06 2024-09-13T09:19:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416883+0000", 2024-09-13T09:19:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.477e-06 2024-09-13T09:19:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416888+0000", 2024-09-13T09:19:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2580000000000003e-06 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477357+0000", 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060469055000000001 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477396+0000", 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9402e-05 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477451+0000", 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4262000000000003e-05 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56104 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5636096~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.629595745000003, 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060778881999999999, 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56104 2024-09-13T09:19:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416892+0000", 2024-09-13T09:19:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416894+0000", 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4049999999999998e-06 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416900+0000", 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.959e-06 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416905+0000", 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-13T09:19:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477465+0000", 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060559460000000002 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477521+0000", 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5813000000000002e-05 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477671+0000", 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150366 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56105 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5734400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.629551534000001, 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060898832, 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56105 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416936+0000", 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416939+0000", 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3550000000000001e-06 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416947+0000", 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0949999999999996e-06 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416953+0000", 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6790000000000002e-06 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477708+0000", 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060755454 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477760+0000", 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1589999999999999e-05 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477835+0000", 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5394000000000004e-05 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56106 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5832704~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.629530216999996, 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061142999000000003, 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56106 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416958+0000", 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416957+0000", 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416960+0000", 2024-09-13T09:19:40.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6979999999999998e-06 2024-09-13T09:19:40.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416967+0000", 2024-09-13T09:19:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6919999999999996e-06 2024-09-13T09:19:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416973+0000", 2024-09-13T09:19:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5679999999999996e-06 2024-09-13T09:19:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477867+0000", 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060894082000000002 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.477924+0000", 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7015999999999998e-05 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478101+0000", 2024-09-13T09:19:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017648300000000001 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56107 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 5931008~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.629201602999998, 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060956639999999999, 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56107 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417286+0000", 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.416977+0000", 2024-09-13T09:19:40.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996905 2024-09-13T09:19:40.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417291+0000", 2024-09-13T09:19:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031392799999999999 2024-09-13T09:19:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417311+0000", 2024-09-13T09:19:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0343000000000002e-05 2024-09-13T09:19:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417329+0000", 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7612e-05 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478123+0000", 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060794421000000001 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478174+0000", 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0614e-05 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478243+0000", 2024-09-13T09:19:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9226000000000004e-05 2024-09-13T09:19:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56108 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 6029312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.629142118999994, 2024-09-13T09:19:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061163618000000003, 2024-09-13T09:19:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56108 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417346+0000", 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417345+0000", 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417349+0000", 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7419999999999999e-06 2024-09-13T09:19:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417367+0000", 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7714999999999999e-05 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.417382+0000", 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5396999999999998e-05 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478270+0000", 2024-09-13T09:19:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060888022999999999 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478322+0000", 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2234999999999997e-05 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.478509+0000", 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187109 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56121 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7307264~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.625158626000001, 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060455939, 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56121 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421329+0000", 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421332+0000", 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421358+0000", 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6427999999999998e-05 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421374+0000", 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6137000000000002e-05 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481604+0000", 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060229528999999997 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481680+0000", 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6089999999999998e-05 2024-09-13T09:19:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481785+0000", 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105259 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56122 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7405568~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.625085123000005, 2024-09-13T09:19:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060604366999999999, 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56122 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421403+0000", 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421402+0000", 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421406+0000", 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.337e-06 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421415+0000", 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421434+0000", 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8617000000000001e-05 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481827+0000", 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060393883000000002 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.481910+0000", 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2657999999999994e-05 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482007+0000", 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7057000000000003e-05 2024-09-13T09:19:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56123 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7503872~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.625048585000002, 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060838125999999999, 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56123 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421439+0000", 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421450+0000", 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1544e-05 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421463+0000", 2024-09-13T09:19:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421468+0000", 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6250000000000004e-06 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482066+0000", 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060597713999999997 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482163+0000", 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7112000000000001e-05 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482277+0000", 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114274 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56124 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7602176~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.625014629999995, 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061054929000000001, 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56124 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421473+0000", 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421476+0000", 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2059999999999999e-06 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421484+0000", 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7889999999999995e-06 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421489+0000", 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:19:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482321+0000", 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060831477000000002 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482403+0000", 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2776999999999997e-05 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482528+0000", 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012491600000000001 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56125 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7700480~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.624994333000004, 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061273721000000003, 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56125 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421494+0000", 2024-09-13T09:19:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421493+0000", 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421496+0000", 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.666e-06 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421503+0000", 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421508+0000", 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0919999999999997e-06 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482575+0000", 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061066339999999997 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482659+0000", 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4158000000000003e-05 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482767+0000", 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010849 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56126 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7798784~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.624974967, 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061489349999999998, 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56126 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421513+0000", 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421531+0000", 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8218000000000001e-05 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421538+0000", 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.345e-06 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421544+0000", 2024-09-13T09:19:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482813+0000", 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061268959999999997 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.482900+0000", 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6858000000000001e-05 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483002+0000", 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102599 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56127 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 7897088~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.624939225999995, 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061729777, 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56127 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421549+0000", 2024-09-13T09:19:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421548+0000", 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421551+0000", 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421559+0000", 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4499999999999998e-06 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.421564+0000", 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483067+0000", 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.061503073999999998 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483153+0000", 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.483278+0000", 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012522499999999999 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56319 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1613824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.619181479999995, 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060599663999999998, 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56319 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427306+0000", 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427309+0000", 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.672e-06 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427316+0000", 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6850000000000001e-06 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427322+0000", 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487756+0000", 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060434817000000002 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487825+0000", 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8553000000000006e-05 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487906+0000", 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1067000000000002e-05 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56320 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1712128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.619161665999997, 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060803930999999999, 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56320 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427326+0000", 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427328+0000", 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427379+0000", 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0343999999999998e-05 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427385+0000", 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3969999999999999e-06 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.487941+0000", 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060555805999999997 2024-09-13T09:19:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488034+0000", 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488130+0000", 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6348999999999995e-05 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56321 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1810432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.619096729000006, 2024-09-13T09:19:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060999937999999997, 2024-09-13T09:19:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56321 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427391+0000", 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427394+0000", 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.721e-06 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427403+0000", 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7110000000000001e-06 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427409+0000", 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9710000000000001e-06 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488183+0000", 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060774003 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488291+0000", 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010786899999999999 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488391+0000", 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100142 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56322 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 1908736~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.618750614000007, 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060864109999999999, 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56322 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427737+0000", 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427433+0000", 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9996958 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427742+0000", 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000309084 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427754+0000", 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1354e-05 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427761+0000", 2024-09-13T09:19:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8059999999999995e-06 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488429+0000", 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060667139000000002 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488496+0000", 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7595000000000006e-05 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488601+0000", 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105257 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:56323 3.19 3:9f6b7de4:::rbd_data.118568665621.0000000000000000:1d [read 2007040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.618144919000002, 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060546618000000003, 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 56323 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428343+0000", 2024-09-13T09:19:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.427767+0000", 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9994245 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428348+0000", 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00058032600000000004 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428360+0000", 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2103000000000001e-05 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.428367+0000", 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1949999999999997e-06 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488650+0000", 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060282626999999998 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488745+0000", 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5365999999999995e-05 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:35.488890+0000", 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000144506 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.146 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:40.146 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:40.147 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:40.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:19:40.148 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:40.149 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:40.149 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:40.140+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:40.247 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:40.240+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:40.325 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:40.325 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:40.325 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:40.325 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:40.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:19:40.349 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:40.345+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:40.450 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:40.444+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:40.517 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:40.518 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:40.532 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:19:40.576 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:40.572+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:40.652 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:40.648+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:40.697 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33548 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.705223508000003, 2024-09-13T09:19:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025760904000000001, 2024-09-13T09:19:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33548 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990124+0000", 2024-09-13T09:19:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:19:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990130+0000", 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990135+0000", 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007826+0000", 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017691018999999999 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007896+0000", 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0059000000000001e-05 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.015883+0000", 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079875269999999995 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33751 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1593344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701803854000005, 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023779101, 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33751 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993548+0000", 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.776e-06 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993555+0000", 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993560+0000", 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017188+0000", 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023627984000000001 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017271+0000", 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2798000000000002e-05 2024-09-13T09:19:40.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017321+0000", 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0557999999999999e-05 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33752 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701781733999994, 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023925173000000001, 2024-09-13T09:19:40.707 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.708 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.708 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.708 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.708 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.708 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33752 2024-09-13T09:19:40.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.708 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:40.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993566+0000", 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993573+0000", 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.759e-06 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993578+0000", 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8339999999999998e-06 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017349+0000", 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023770959000000001 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017423+0000", 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3652000000000006e-05 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017489+0000", 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6748999999999994e-05 2024-09-13T09:19:40.711 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.712 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.712 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.712 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33753 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1650688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.712 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:40.712 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701763252000006, 2024-09-13T09:19:40.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024066349000000001, 2024-09-13T09:19:40.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.712 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.712 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.712 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.713 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.713 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33753 2024-09-13T09:19:40.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.713 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:40.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993582+0000", 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993585+0000", 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993591+0000", 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993595+0000", 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017515+0000", 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023920117000000001 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017583+0000", 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8485999999999994e-05 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017649+0000", 2024-09-13T09:19:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33754 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701746958000001, 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02420106, 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.717 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33754 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:40.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993601+0000", 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993607+0000", 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9669999999999998e-06 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993611+0000", 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017669+0000", 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024057683999999999 2024-09-13T09:19:40.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017742+0000", 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3397000000000002e-05 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017800+0000", 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7711999999999999e-05 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33755 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1708032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701730538999996, 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024346923999999999, 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33755 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993617+0000", 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993624+0000", 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0469999999999999e-06 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993628+0000", 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017830+0000", 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024201879999999999 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017897+0000", 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7096999999999998e-05 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017962+0000", 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5455000000000003e-05 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33756 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701713812999998, 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024485512000000001, 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.726 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33756 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993635+0000", 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:19:40.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993641+0000", 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993645+0000", 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2520000000000001e-06 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017979+0000", 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024333950999999999 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018045+0000", 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5843e-05 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018118+0000", 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2417999999999997e-05 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.730 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33757 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1765376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701696415000001, 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024605578999999999, 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33757 2024-09-13T09:19:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993649+0000", 2024-09-13T09:19:40.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993651+0000", 2024-09-13T09:19:40.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.841e-06 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993657+0000", 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993662+0000", 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018139+0000", 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024476748 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018196+0000", 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7188000000000002e-05 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018255+0000", 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9045999999999997e-05 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.735 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33758 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701679781999999, 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024736898, 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.736 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33758 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993668+0000", 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993674+0000", 2024-09-13T09:19:40.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993679+0000", 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.515e-06 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018284+0000", 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024604816000000002 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018356+0000", 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2782999999999998e-05 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018403+0000", 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6726e-05 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33759 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1822720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.740 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701663175999997, 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873635000000002, 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33759 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993682+0000", 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993685+0000", 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993691+0000", 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1840000000000004e-06 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993696+0000", 2024-09-13T09:19:40.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018429+0000", 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024733330000000001 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018494+0000", 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5233000000000004e-05 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018556+0000", 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2125999999999994e-05 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33760 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701646320999998, 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025058980000000002, 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.745 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33760 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:40.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993699+0000", 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993702+0000", 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.632e-06 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993708+0000", 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2280000000000003e-06 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993713+0000", 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018586+0000", 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873572 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018645+0000", 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9098000000000002e-05 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018759+0000", 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113224 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33761 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1880064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701629624000006, 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025229076, 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33761 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993718+0000", 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.092e-06 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993724+0000", 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0839999999999997e-06 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993729+0000", 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018790+0000", 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025061039 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018871+0000", 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0885999999999998e-05 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018945+0000", 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4773999999999999e-05 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33762 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.70161315, 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025559609, 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.754 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33762 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993732+0000", 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993735+0000", 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993740+0000", 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993745+0000", 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5290000000000002e-06 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019043+0000", 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025298355000000002 2024-09-13T09:19:40.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019134+0000", 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0781999999999999e-05 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019292+0000", 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015829 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33763 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701597208999999, 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025732657999999999, 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33763 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993748+0000", 2024-09-13T09:19:40.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993755+0000", 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993762+0000", 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993767+0000", 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7230000000000004e-06 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019322+0000", 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025555564999999999 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019402+0000", 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9433999999999999e-05 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019481+0000", 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9739000000000003e-05 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33764 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701574835000002, 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025908526000000001, 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.763 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33764 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993773+0000", 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993779+0000", 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9610000000000002e-06 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993783+0000", 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.459e-06 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019513+0000", 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025729399 2024-09-13T09:19:40.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019609+0000", 2024-09-13T09:19:40.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6504999999999998e-05 2024-09-13T09:19:40.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019680+0000", 2024-09-13T09:19:40.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0393000000000002e-05 2024-09-13T09:19:40.767 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.767 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.767 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33765 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1994752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701558599999998, 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026051009999999999, 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33765 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.768 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993789+0000", 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993795+0000", 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.088e-06 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993799+0000", 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019704+0000", 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025904311999999999 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019766+0000", 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2829000000000004e-05 2024-09-13T09:19:40.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019838+0000", 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1935999999999997e-05 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33766 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701542544999995, 2024-09-13T09:19:40.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026238971999999999, 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33766 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993805+0000", 2024-09-13T09:19:40.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993811+0000", 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9000000000000003e-06 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993816+0000", 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.527e-06 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019877+0000", 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026061337 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019960+0000", 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3245999999999995e-05 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020042+0000", 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2053999999999997e-05 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33767 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701526134999995, 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026407013999999999, 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33767 2024-09-13T09:19:40.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993819+0000", 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993822+0000", 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993828+0000", 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993832+0000", 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.493e-06 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020072+0000", 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026239519999999999 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020151+0000", 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9278999999999998e-05 2024-09-13T09:19:40.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020227+0000", 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5965999999999997e-05 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33768 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.701507681999999, 2024-09-13T09:19:40.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026564424, 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33768 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993840+0000", 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.996e-06 2024-09-13T09:19:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993846+0000", 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993858+0000", 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2214e-05 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020253+0000", 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026394926999999999 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020329+0000", 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5193999999999999e-05 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020403+0000", 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4145000000000002e-05 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:78829 3.6 3:6418b130:::rbd_data.11853dd604c.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.478821488000001, 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041141776999999997, 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 78829 2024-09-13T09:19:40.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216522+0000", 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216540+0000", 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8280000000000001e-05 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216577+0000", 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7151999999999999e-05 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:40.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216594+0000", 2024-09-13T09:19:40.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7113e-05 2024-09-13T09:19:40.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:40.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216696+0000", 2024-09-13T09:19:40.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010190400000000001 2024-09-13T09:19:40.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:40.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216833+0000", 2024-09-13T09:19:40.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136697 2024-09-13T09:19:40.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:40.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.224657+0000", 2024-09-13T09:19:40.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078241000000000005 2024-09-13T09:19:40.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257365+0000", 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032707984000000002 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257404+0000", 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8979000000000001e-05 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:40.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257549+0000", 2024-09-13T09:19:40.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000145737 2024-09-13T09:19:40.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:40.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:40.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:40.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257666+0000", 2024-09-13T09:19:40.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116754 2024-09-13T09:19:40.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.792 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:40.792 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:40.792 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:40.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:19:40.794 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:40.792+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:40.895 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:40.893+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:40.975 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:40.976 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:40.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:19:41.036 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:41.032+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:41.097 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:41.093+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:41.131 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:41.131 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:41.131 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:41.131 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:41.131 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:41.147 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:19:41.232 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:41.228+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:41.298 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:41.297+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:41.320 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:41.320 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:41.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:41.321 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:41.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.321 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:19:41.321 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:41.321 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 113.780987907, 2024-09-13T09:19:41.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:19:41.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:41.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:19:41.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:19:41.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411420922000005, 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.327 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:19:41.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:19:41.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411403601000004, 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:19:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:19:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:19:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411386660000005, 2024-09-13T09:19:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:19:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411369432000001, 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411333010999996, 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411279549, 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411257528999997, 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411232893999994, 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411214371, 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:19:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411196500000003, 2024-09-13T09:19:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:19:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:19:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:19:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:19:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:19:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:19:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:19:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:19:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:19:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:19:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411179169999997, 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:19:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:19:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411161699999994, 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:19:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411144833999998, 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411127046999994, 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:19:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.411082737000001, 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:19:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:19:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:19:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.177952249000001, 2024-09-13T09:19:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:19:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:19:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.177935415999997, 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:19:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.177919058000001, 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:19:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:66592 3.7 3:e0817996:::rbd_data.1185b1e141e2.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.673559328000003, 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.051168777999999998, 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 66592 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643787+0000", 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643799+0000", 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2140000000000001e-05 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643827+0000", 2024-09-13T09:19:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8121999999999999e-05 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643841+0000", 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3969e-05 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643985+0000", 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014404199999999999 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.644371+0000", 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038588700000000001 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.654317+0000", 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099452050000000004 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694641+0000", 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.040324802999999999 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694682+0000", 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0451999999999998e-05 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694828+0000", 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014580499999999999 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694958+0000", 2024-09-13T09:19:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013029599999999999 2024-09-13T09:19:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:41.408 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:41.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:19:41.410 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:41.404+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:41.510 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:41.509+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:41.544 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:41.544 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:41.544 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:41.544 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:41.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:19:41.650 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:41.649+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:41.714 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:41.713+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:41.728 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:41.729 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:41.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:19:41.815 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:41.808+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:41.916 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:41.912+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:41.929 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.939787563, 2024-09-13T09:19:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:19:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:19:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:19:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:19:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:19:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:19:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:19:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:19:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:19:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:19:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:19:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:19:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:19:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:19:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.939738827, 2024-09-13T09:19:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:19:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:19:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:19:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:19:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:19:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:19:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:19:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:19:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.939671158, 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:19:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:19:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:19:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:19:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:19:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:19:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:19:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:19:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:19:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.939530949, 2024-09-13T09:19:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:19:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:19:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:19:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:19:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:19:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:19:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.93944008299999, 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:19:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:19:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:19:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:19:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:19:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:19:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:19:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.939365223, 2024-09-13T09:19:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:19:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:19:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:19:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:19:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:19:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:19:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:19:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:19:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:19:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:19:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:19:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:19:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:19:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:19:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:19:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.939294047, 2024-09-13T09:19:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:19:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.961 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:19:41.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.961 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:41.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:41.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:19:41.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:19:41.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:19:41.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:19:41.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:19:41.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:19:41.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:19:41.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:19:41.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:19:41.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:19:41.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:19:41.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.93924499400001, 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.966 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.966 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.966 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:19:41.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.966 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:41.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:19:41.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:19:41.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:19:41.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:19:41.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:19:41.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:19:41.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:19:41.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:19:41.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:19:41.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:19:41.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.939183258, 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:41.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:19:41.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:19:41.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:19:41.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.975 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.975 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.975 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:41.975 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.939133588, 2024-09-13T09:19:41.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:19:41.975 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.975 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.975 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.975 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:19:41.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.976 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:41.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:19:41.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:19:41.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:19:41.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:19:41.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.979 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.980 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.980 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:41.980 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.939085069, 2024-09-13T09:19:41.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:19:41.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.981 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:19:41.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.981 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:41.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:41.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:19:41.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:19:41.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:19:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:19:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:19:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:19:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:19:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:19:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.939036673, 2024-09-13T09:19:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:19:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:41.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:19:41.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:19:41.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:19:41.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:19:41.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:19:41.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:19:41.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:19:41.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:19:41.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:19:41.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.990 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.990 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.990 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:41.990 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.938989386, 2024-09-13T09:19:41.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:19:41.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:19:41.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.991 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:41.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:41.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:19:41.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:19:41.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:19:41.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:19:41.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:19:41.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:19:41.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:19:41.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:19:41.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:19:41.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:19:41.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:19:41.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:41.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:19:41.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:19:41.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.995 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:41.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:41.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.995 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:41.996 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:41.996 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.938905984, 2024-09-13T09:19:41.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:19:41.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:41.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:41.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:41.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:41.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:41.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:19:41.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.996 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:41.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:41.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:41.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:41.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:41.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:41.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:41.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:19:41.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:41.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:41.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:19:41.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:19:41.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:41.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:19:41.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:19:41.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:41.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:42.000 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:42.001 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.93883225800001, 2024-09-13T09:19:42.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:19:42.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:42.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:42.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:42.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:42.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:42.001 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:19:42.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.001 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:42.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:19:42.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:19:42.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:42.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:19:42.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:19:42.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:42.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:19:42.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:19:42.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:42.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:19:42.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:19:42.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.005 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:42.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.005 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:42.005 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:42.005 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.938764135, 2024-09-13T09:19:42.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:42.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:42.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:19:42.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:19:42.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:42.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:19:42.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:19:42.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:42.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:19:42.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:19:42.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:42.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.93871366800001, 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:42.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:42.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:42.011 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:19:42.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.011 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:42.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:42.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:42.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:42.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:42.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:19:42.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:42.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.015 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:78833 3.a 3:5235bc0b:::rbd_data.11853dd604c.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:42.015 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:42.015 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.656443850999999, 2024-09-13T09:19:42.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.35280560599999999, 2024-09-13T09:19:42.015 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:42.015 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:42.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:42.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:42.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:42.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78833 2024-09-13T09:19:42.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.016 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:42.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:42.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:42.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:42.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:42.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:42.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:42.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:42.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269309+0000", 2024-09-13T09:19:42.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:19:42.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:42.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269322+0000", 2024-09-13T09:19:42.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3359e-05 2024-09-13T09:19:42.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269352+0000", 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0320000000000001e-05 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269367+0000", 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.526e-05 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269434+0000", 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.648e-05 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269446+0000", 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2286999999999999e-05 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.606711+0000", 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.33726426399999998 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.606879+0000", 2024-09-13T09:19:42.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016884100000000001 2024-09-13T09:19:42.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-13T09:19:42.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.609371+0000", 2024-09-13T09:19:42.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024920110000000001 2024-09-13T09:19:42.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:42.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622034+0000", 2024-09-13T09:19:42.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012662199000000001 2024-09-13T09:19:42.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622042+0000", 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9570000000000002e-06 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622083+0000", 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.197e-05 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622117+0000", 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3013999999999999e-05 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:80090 3.a 3:517b773f:::rbd_object_map.118553778f1e:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.200070555000003, 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99537752499999999, 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:42.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:42.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80090 2024-09-13T09:19:42.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.024 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:42.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:42.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:42.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:42.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:42.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:42.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:42.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:42.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725681+0000", 2024-09-13T09:19:42.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-13T09:19:42.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:42.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725699+0000", 2024-09-13T09:19:42.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7683999999999999e-05 2024-09-13T09:19:42.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:42.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725746+0000", 2024-09-13T09:19:42.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7196e-05 2024-09-13T09:19:42.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:42.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725769+0000", 2024-09-13T09:19:42.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3383e-05 2024-09-13T09:19:42.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:42.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725956+0000", 2024-09-13T09:19:42.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000187049 2024-09-13T09:19:42.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:42.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725985+0000", 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8472000000000001e-05 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.718483+0000", 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99249815200000002 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.718851+0000", 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036844299999999998 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:42.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.719201+0000", 2024-09-13T09:19:42.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034928699999999999 2024-09-13T09:19:42.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:42.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720843+0000", 2024-09-13T09:19:42.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016425249999999999 2024-09-13T09:19:42.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:42.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720882+0000", 2024-09-13T09:19:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9045999999999999e-05 2024-09-13T09:19:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720985+0000", 2024-09-13T09:19:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103163 2024-09-13T09:19:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.721062+0000", 2024-09-13T09:19:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6402000000000005e-05 2024-09-13T09:19:42.030 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:81122 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.025230142000002, 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.920409343, 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:42.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 81122 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:42.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900523+0000", 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900536+0000", 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3200000000000001e-05 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900565+0000", 2024-09-13T09:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9703e-05 2024-09-13T09:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900588+0000", 2024-09-13T09:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2096e-05 2024-09-13T09:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900698+0000", 2024-09-13T09:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110029 2024-09-13T09:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900716+0000", 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8070000000000001e-05 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:17.766722+0000", 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.86600628099999999 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:42.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:17.766770+0000", 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8486000000000003e-05 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.817359+0000", 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.050588836 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.817852+0000", 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00049274599999999996 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-13T09:19:42.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.818512+0000", 2024-09-13T09:19:42.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00066035099999999999 2024-09-13T09:19:42.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:42.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820216+0000", 2024-09-13T09:19:42.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017038680000000001 2024-09-13T09:19:42.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:42.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820601+0000", 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038506599999999999 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820746+0000", 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014458900000000001 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820934+0000", 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188102 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:42.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:42.039 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:42.039 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:42.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:19:42.041 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:19:42.041 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:42.040+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:42.141 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:42.136+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:42.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:19:42.172 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:42.172 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:42.172 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:42.172 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:42.186 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:19:42.242 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:42.236+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:42.303 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:42.303 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:42.306 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:42.306 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:42.306 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:42.319 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:42.319 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:42.321 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:19:42.335 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:42.335 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:42.351 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:42.365 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:42.365+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:42.368 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:42.387 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:42.404 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:42.425 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:42.441 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:42.441 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:42.444 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:42.441+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:42.456 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:19:42.544 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:42.540+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:42.622 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:42.622 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:42.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:19:42.645 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:42.644+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:42.746 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:42.744+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:42.763 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:42.763 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:42.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:19:42.846 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:42.840+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:42.903 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:42.903 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:42.917 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:19:42.947 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:42.940+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-13T09:19:43.049 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:43.049+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:43.058 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:43.058 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:43.075 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:19:43.150 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:43.144+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:43.245 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:43.245 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:43.252 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:43.249+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:43.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:19:43.352 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:43.344+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:43.430 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:43.430 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:43.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:19:43.492 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:43.488+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:43.553 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:43.548+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:43.579 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:43.579 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:43.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:19:43.654 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:43.653+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:43.755 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:43.748+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:43.763 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:43.763 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:43.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:19:43.869 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:43.864+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:43.945 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:43.945 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:43.957 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:43.953+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:43.960 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:19:44.058 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:44.052+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:44.137 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:44.137 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:44.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:19:44.196 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:44.192+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:44.259 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:44.252+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:44.320 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:44.320 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:44.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:19:44.353 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:19:44.353 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:19:44.353 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T09:19:44.353 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:19:44.360 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:44.357+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:44.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:19:44.461 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:44.457+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:44.501 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:44.501 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:44.518 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:19:44.564 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:44.560+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:44.602 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:44.620 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:44.636 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:44.637 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:44.637 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:44.637 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:44.638 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:44.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:19:44.654 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:44.675 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:44.675 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:44.668+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:44.693 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:19:44.693 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:44.693 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 141, 2024-09-13T09:19:44.693 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "f0889cb1-03d6-4b81-88c9-6141846c6033", 2024-09-13T09:19:44.693 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-13T09:16:56.964982+0000", 2024-09-13T09:19:44.693 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-13T09:19:43.958097+0000", 2024-09-13T09:19:44.693 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-13T09:17:08.368378+0000", 2024-09-13T09:19:44.693 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-13T09:19:16.749126+0000", 2024-09-13T09:19:44.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:19:44.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5806080, 2024-09-13T09:19:44.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: "nodeep-scrub", 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: "noscrub", 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-13T09:19:44.694 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-13T09:17:10.140920+0000", 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-13T09:19:44.695 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-13T09:19:44.696 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "17", 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:19:44.697 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-13T09:19:44.698 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-13T09:19:44.699 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-13T09:19:44.700 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 6.9000000953674316, 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 6.9000000953674316, 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.28999999165534973, 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.86565208435058594, 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-13T09:17:15.775544+0000", 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:19:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-13T09:19:44.703 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:19:44.703 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-13T09:19:44.703 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-13T09:19:44.703 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-13T09:19:44.703 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-13T09:19:44.703 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-13T09:19:44.703 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-13T09:19:44.703 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-13T09:19:44.703 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "21", 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-13T09:19:44.704 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 21, 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:19:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:19:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-09-13T09:19:44.707 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.86565208435058594, 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-13T09:17:25.804931+0000", 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-13T09:19:44.708 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-13T09:19:44.709 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-13T09:19:44.710 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-13T09:19:44.710 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-13T09:19:44.710 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-13T09:19:44.710 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-13T09:19:44.710 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-13T09:19:44.710 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-13T09:19:44.710 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-13T09:19:44.710 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-13T09:19:44.710 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-13T09:19:44.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.710 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "141", 2024-09-13T09:19:44.710 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 75, 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 141, 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-13T09:19:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-13T09:19:44.712 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-13T09:19:44.712 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-13T09:19:44.712 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-13T09:19:44.712 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-13T09:19:44.712 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:19:44.712 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:19:44.712 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:19:44.712 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-13T09:19:44.712 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-13T09:19:44.712 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-13T09:19:44.712 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-13T09:19:44.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-13T09:19:44.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.713 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-13T09:19:44.713 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-13T09:19:44.713 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-13T09:19:44.713 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:19:44.713 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:19:44.713 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:19:44.713 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:19:44.713 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-13T09:19:44.713 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-13T09:19:44.713 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-13T09:19:44.713 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.3200000524520874, 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.3200000524520874, 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1.1399999856948853, 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.86565208435058594, 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-13T09:19:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "786bdfc4-cc25-4167-aa5b-3737cfeb4ac1", 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:44.715 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:44.716 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 116, 2024-09-13T09:19:44.716 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:44.716 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:44.716 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:44.716 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.716 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-13T09:19:44.716 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:44.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.717 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-13T09:19:44.717 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:44.717 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.717 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.717 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:44.717 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3549111487 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/3549111487", 2024-09-13T09:19:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/3549111487", 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/3549111487", 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/3549111487", 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "ca6189ab-90dd-4e9d-a85f-092186153b5a", 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:44.721 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 114, 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:44.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-13T09:19:44.723 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.724 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1470371899 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6810/1470371899", 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6815/1470371899", 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1470371899", 2024-09-13T09:19:44.726 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6820/1470371899", 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a", 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0, 2024-09-13T09:19:44.727 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-13T09:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-13T09:19:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2485123706 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/2485123706", 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/2485123706", 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/2485123706", 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/2485123706", 2024-09-13T09:19:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218", 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 125, 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1009890408 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6811/1009890408", 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6814/1009890408", 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6821/1009890408", 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6818/1009890408", 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39f42e87-29dd-42cc-9ef6-e0074f1794e4", 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 117, 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2723023689 2024-09-13T09:19:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6823/2723023689", 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6825/2723023689", 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/2723023689", 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6828/2723023689", 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "cf8dd701-9543-4bdd-a6f6-0b56210f00c2", 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2955019640 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6809/2955019640", 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6812/2955019640", 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6819/2955019640", 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6815/2955019640", 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "dd080b88-3292-4a34-8dbd-645683d8fe14", 2024-09-13T09:19:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.9252166748046875, 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 131, 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-13T09:19:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-13T09:19:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 916160151 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6813/916160151", 2024-09-13T09:19:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6818/916160151", 2024-09-13T09:19:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6829/916160151", 2024-09-13T09:19:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6822/916160151", 2024-09-13T09:19:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-13T09:19:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c6bca54b-e982-4d6a-9ae9-057ee3881a5b", 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 101, 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-13T09:19:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-13T09:19:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3477167689 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6801/3477167689", 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6803/3477167689", 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6807/3477167689", 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6805/3477167689", 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-13T09:19:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.247113+0000", 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.166872+0000", 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.451769+0000", 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.458130+0000", 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.466129+0000", 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.444828+0000", 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.425145+0000", 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-13T09:17:05.484371+0000", 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-09-13T09:19:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 10, 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 5 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [ 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 10, 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-13T09:19:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-13T09:19:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:19:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:19:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-13T09:19:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:44.775 INFO:tasks.thrashosds.thrasher:j is {'epoch': 141, 'fsid': 'f0889cb1-03d6-4b81-88c9-6141846c6033', 'created': '2024-09-13T09:16:56.964982+0000', 'modified': '2024-09-13T09:19:43.958097+0000', 'last_up_change': '2024-09-13T09:17:08.368378+0000', 'last_in_change': '2024-09-13T09:19:16.749126+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:17:10.140920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8656520843505859, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:17:15.775544+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8656520843505859, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:17:25.804931+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 75, 'snap_epoch': 141, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8656520843505859, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.8700000047683716}}], 'osds': [{'osd': 0, 'uuid': '786bdfc4-cc25-4167-aa5b-3737cfeb4ac1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 3549111487}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 3549111487}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 3549111487}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 3549111487}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 3549111487}]}, 'public_addr': '172.21.15.32:6801/3549111487', 'cluster_addr': '172.21.15.32:6803/3549111487', 'heartbeat_back_addr': '172.21.15.32:6807/3549111487', 'heartbeat_front_addr': '172.21.15.32:6805/3549111487', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'ca6189ab-90dd-4e9d-a85f-092186153b5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 114, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6810', 'nonce': 1470371899}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 1470371899}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1470371899}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6817', 'nonce': 1470371899}, {'type': 'v1', 'addr': '172.21.15.32:6820', 'nonce': 1470371899}]}, 'public_addr': '172.21.15.32:6810/1470371899', 'cluster_addr': '172.21.15.32:6815/1470371899', 'heartbeat_back_addr': '172.21.15.32:6823/1470371899', 'heartbeat_front_addr': '172.21.15.32:6820/1470371899', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 2485123706}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 2485123706}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 2485123706}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 2485123706}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 2485123706}]}, 'public_addr': '172.21.15.32:6825/2485123706', 'cluster_addr': '172.21.15.32:6827/2485123706', 'heartbeat_back_addr': '172.21.15.32:6831/2485123706', 'heartbeat_front_addr': '172.21.15.32:6829/2485123706', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 125, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6809', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 1009890408}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6813', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6814', 'nonce': 1009890408}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6819', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1009890408}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1009890408}, {'type': 'v1', 'addr': '172.21.15.32:6818', 'nonce': 1009890408}]}, 'public_addr': '172.21.15.32:6811/1009890408', 'cluster_addr': '172.21.15.32:6814/1009890408', 'heartbeat_back_addr': '172.21.15.32:6821/1009890408', 'heartbeat_front_addr': '172.21.15.32:6818/1009890408', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39f42e87-29dd-42cc-9ef6-e0074f1794e4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6823', 'nonce': 2723023689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6824', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 2723023689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 2723023689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 2723023689}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 2723023689}]}, 'public_addr': '172.21.15.46:6823/2723023689', 'cluster_addr': '172.21.15.46:6825/2723023689', 'heartbeat_back_addr': '172.21.15.46:6831/2723023689', 'heartbeat_front_addr': '172.21.15.46:6828/2723023689', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'cf8dd701-9543-4bdd-a6f6-0b56210f00c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6809', 'nonce': 2955019640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6811', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6812', 'nonce': 2955019640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 2955019640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6814', 'nonce': 2955019640}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 2955019640}]}, 'public_addr': '172.21.15.46:6809/2955019640', 'cluster_addr': '172.21.15.46:6812/2955019640', 'heartbeat_back_addr': '172.21.15.46:6819/2955019640', 'heartbeat_front_addr': '172.21.15.46:6815/2955019640', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'dd080b88-3292-4a34-8dbd-645683d8fe14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9252166748046875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 131, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6810', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6813', 'nonce': 916160151}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 916160151}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 916160151}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 916160151}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 916160151}]}, 'public_addr': '172.21.15.46:6813/916160151', 'cluster_addr': '172.21.15.46:6818/916160151', 'heartbeat_back_addr': '172.21.15.46:6829/916160151', 'heartbeat_front_addr': '172.21.15.46:6822/916160151', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c6bca54b-e982-4d6a-9ae9-057ee3881a5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6801', 'nonce': 3477167689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6802', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 3477167689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6806', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 3477167689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 3477167689}, {'type': 'v1', 'addr': '172.21.15.46:6805', 'nonce': 3477167689}]}, 'public_addr': '172.21.15.46:6801/3477167689', 'cluster_addr': '172.21.15.46:6803/3477167689', 'heartbeat_back_addr': '172.21.15.46:6807/3477167689', 'heartbeat_front_addr': '172.21.15.46:6805/3477167689', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.247113+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.166872+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.451769+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.458130+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.466129+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.444828+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.425145+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:17:05.484371+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 10, 'length': 5}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 10, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:19:44.775 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:19:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:44.778 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:44.773+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:44.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:19:44.874 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:44.873+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:44.944 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:44.944 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:44.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:44.944 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:44.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.944 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33548 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:44.945 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:44.945 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.952292087000004, 2024-09-13T09:19:44.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025760904000000001, 2024-09-13T09:19:44.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:44.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:44.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:44.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:44.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:44.945 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33548 2024-09-13T09:19:44.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.945 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:44.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:44.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:44.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:44.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990124+0000", 2024-09-13T09:19:44.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:19:44.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:44.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990130+0000", 2024-09-13T09:19:44.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:19:44.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:44.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990135+0000", 2024-09-13T09:19:44.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:19:44.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:44.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007826+0000", 2024-09-13T09:19:44.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017691018999999999 2024-09-13T09:19:44.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:44.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007896+0000", 2024-09-13T09:19:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0059000000000001e-05 2024-09-13T09:19:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.015883+0000", 2024-09-13T09:19:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079875269999999995 2024-09-13T09:19:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33751 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1593344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948872433000005, 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023779101, 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33751 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:44.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993548+0000", 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.776e-06 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993555+0000", 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:44.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993560+0000", 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017188+0000", 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023627984000000001 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017271+0000", 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2798000000000002e-05 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017321+0000", 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0557999999999999e-05 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33752 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948850312999994, 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023925173000000001, 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:44.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33752 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993566+0000", 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:19:44.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993573+0000", 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.759e-06 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993578+0000", 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8339999999999998e-06 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017349+0000", 2024-09-13T09:19:44.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023770959000000001 2024-09-13T09:19:44.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:44.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017423+0000", 2024-09-13T09:19:44.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3652000000000006e-05 2024-09-13T09:19:44.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:44.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017489+0000", 2024-09-13T09:19:44.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6748999999999994e-05 2024-09-13T09:19:44.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33753 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1650688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948831831000007, 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024066349000000001, 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:44.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33753 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:44.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993582+0000", 2024-09-13T09:19:44.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:44.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993585+0000", 2024-09-13T09:19:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:19:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993591+0000", 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993595+0000", 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017515+0000", 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023920117000000001 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017583+0000", 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8485999999999994e-05 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017649+0000", 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33754 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948815537000002, 2024-09-13T09:19:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02420106, 2024-09-13T09:19:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33754 2024-09-13T09:19:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993601+0000", 2024-09-13T09:19:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:19:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993607+0000", 2024-09-13T09:19:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9669999999999998e-06 2024-09-13T09:19:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993611+0000", 2024-09-13T09:19:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:19:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017669+0000", 2024-09-13T09:19:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024057683999999999 2024-09-13T09:19:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017742+0000", 2024-09-13T09:19:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3397000000000002e-05 2024-09-13T09:19:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017800+0000", 2024-09-13T09:19:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7711999999999999e-05 2024-09-13T09:19:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33755 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1708032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948799117999997, 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024346923999999999, 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33755 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993617+0000", 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993624+0000", 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0469999999999999e-06 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993628+0000", 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017830+0000", 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024201879999999999 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017897+0000", 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7096999999999998e-05 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017962+0000", 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5455000000000003e-05 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33756 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948782391999998, 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024485512000000001, 2024-09-13T09:19:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33756 2024-09-13T09:19:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993635+0000", 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993641+0000", 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993645+0000", 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2520000000000001e-06 2024-09-13T09:19:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017979+0000", 2024-09-13T09:19:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024333950999999999 2024-09-13T09:19:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018045+0000", 2024-09-13T09:19:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5843e-05 2024-09-13T09:19:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018118+0000", 2024-09-13T09:19:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2417999999999997e-05 2024-09-13T09:19:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33757 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1765376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948764994000001, 2024-09-13T09:19:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024605578999999999, 2024-09-13T09:19:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33757 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993649+0000", 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993651+0000", 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.841e-06 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993657+0000", 2024-09-13T09:19:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993662+0000", 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018139+0000", 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024476748 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018196+0000", 2024-09-13T09:19:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7188000000000002e-05 2024-09-13T09:19:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018255+0000", 2024-09-13T09:19:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9045999999999997e-05 2024-09-13T09:19:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33758 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948748361, 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024736898, 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33758 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993668+0000", 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993674+0000", 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993679+0000", 2024-09-13T09:19:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.515e-06 2024-09-13T09:19:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018284+0000", 2024-09-13T09:19:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024604816000000002 2024-09-13T09:19:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018356+0000", 2024-09-13T09:19:45.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2782999999999998e-05 2024-09-13T09:19:45.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018403+0000", 2024-09-13T09:19:45.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6726e-05 2024-09-13T09:19:45.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.024 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.024 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33759 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1822720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948731754999997, 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873635000000002, 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33759 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993682+0000", 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993685+0000", 2024-09-13T09:19:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:19:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993691+0000", 2024-09-13T09:19:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1840000000000004e-06 2024-09-13T09:19:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993696+0000", 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018429+0000", 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024733330000000001 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018494+0000", 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5233000000000004e-05 2024-09-13T09:19:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018556+0000", 2024-09-13T09:19:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2125999999999994e-05 2024-09-13T09:19:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33760 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948714899999999, 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025058980000000002, 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33760 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993699+0000", 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993702+0000", 2024-09-13T09:19:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.632e-06 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993708+0000", 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2280000000000003e-06 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993713+0000", 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018586+0000", 2024-09-13T09:19:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873572 2024-09-13T09:19:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018645+0000", 2024-09-13T09:19:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9098000000000002e-05 2024-09-13T09:19:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018759+0000", 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113224 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33761 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1880064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948698203000006, 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025229076, 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33761 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993718+0000", 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.092e-06 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993724+0000", 2024-09-13T09:19:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0839999999999997e-06 2024-09-13T09:19:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993729+0000", 2024-09-13T09:19:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:19:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018790+0000", 2024-09-13T09:19:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025061039 2024-09-13T09:19:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018871+0000", 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0885999999999998e-05 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018945+0000", 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4773999999999999e-05 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33762 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948681729, 2024-09-13T09:19:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025559609, 2024-09-13T09:19:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33762 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993732+0000", 2024-09-13T09:19:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993735+0000", 2024-09-13T09:19:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:19:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993740+0000", 2024-09-13T09:19:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993745+0000", 2024-09-13T09:19:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5290000000000002e-06 2024-09-13T09:19:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019043+0000", 2024-09-13T09:19:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025298355000000002 2024-09-13T09:19:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019134+0000", 2024-09-13T09:19:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0781999999999999e-05 2024-09-13T09:19:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019292+0000", 2024-09-13T09:19:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015829 2024-09-13T09:19:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33763 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948665788, 2024-09-13T09:19:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025732657999999999, 2024-09-13T09:19:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33763 2024-09-13T09:19:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993748+0000", 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993755+0000", 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993762+0000", 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993767+0000", 2024-09-13T09:19:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7230000000000004e-06 2024-09-13T09:19:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019322+0000", 2024-09-13T09:19:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025555564999999999 2024-09-13T09:19:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019402+0000", 2024-09-13T09:19:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9433999999999999e-05 2024-09-13T09:19:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019481+0000", 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9739000000000003e-05 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33764 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948643414000003, 2024-09-13T09:19:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025908526000000001, 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33764 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993773+0000", 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993779+0000", 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9610000000000002e-06 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993783+0000", 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.459e-06 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019513+0000", 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025729399 2024-09-13T09:19:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019609+0000", 2024-09-13T09:19:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6504999999999998e-05 2024-09-13T09:19:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019680+0000", 2024-09-13T09:19:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0393000000000002e-05 2024-09-13T09:19:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33765 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1994752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948627178999999, 2024-09-13T09:19:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026051009999999999, 2024-09-13T09:19:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33765 2024-09-13T09:19:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993789+0000", 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993795+0000", 2024-09-13T09:19:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.088e-06 2024-09-13T09:19:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993799+0000", 2024-09-13T09:19:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:19:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019704+0000", 2024-09-13T09:19:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025904311999999999 2024-09-13T09:19:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019766+0000", 2024-09-13T09:19:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2829000000000004e-05 2024-09-13T09:19:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019838+0000", 2024-09-13T09:19:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1935999999999997e-05 2024-09-13T09:19:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33766 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948611123999996, 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026238971999999999, 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33766 2024-09-13T09:19:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993805+0000", 2024-09-13T09:19:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:19:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993811+0000", 2024-09-13T09:19:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9000000000000003e-06 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993816+0000", 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.527e-06 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019877+0000", 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026061337 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019960+0000", 2024-09-13T09:19:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3245999999999995e-05 2024-09-13T09:19:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020042+0000", 2024-09-13T09:19:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2053999999999997e-05 2024-09-13T09:19:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33767 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948594713999995, 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026407013999999999, 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33767 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993819+0000", 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993822+0000", 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993828+0000", 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993832+0000", 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.493e-06 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020072+0000", 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026239519999999999 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020151+0000", 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9278999999999998e-05 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020227+0000", 2024-09-13T09:19:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5965999999999997e-05 2024-09-13T09:19:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33768 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.948576260999999, 2024-09-13T09:19:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026564424, 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33768 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993840+0000", 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.996e-06 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993846+0000", 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993858+0000", 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2214e-05 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020253+0000", 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026394926999999999 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020329+0000", 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5193999999999999e-05 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020403+0000", 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4145000000000002e-05 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:78829 3.6 3:6418b130:::rbd_data.11853dd604c.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.725890067000002, 2024-09-13T09:19:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041141776999999997, 2024-09-13T09:19:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 78829 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216522+0000", 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216540+0000", 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8280000000000001e-05 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216577+0000", 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7151999999999999e-05 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216594+0000", 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7113e-05 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216696+0000", 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010190400000000001 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216833+0000", 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136697 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.224657+0000", 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078241000000000005 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257365+0000", 2024-09-13T09:19:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032707984000000002 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257404+0000", 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8979000000000001e-05 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257549+0000", 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000145737 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257666+0000", 2024-09-13T09:19:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116754 2024-09-13T09:19:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.079 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:45.080 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:19:45.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:19:45.082 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:45.076+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:45.131 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:19:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:19:45.156 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-13T09:19:44.973474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88624897,"num_objects":390,"num_object_clones":60,"num_object_copies":780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":390,"num_whiteouts":203,"num_read":125493,"num_read_kb":554506,"num_write":18965,"num_write_kb":381489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1385,"num_bytes_recovered":29874359,"num_keys_recovered":250,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16332,"ondisk_log_size":16332,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":363812,"kb_used_data":22188,"kb_used_omap":12,"kb_used_meta":341491,"kb_avail":754610908,"statfs":{"total":773094113280,"available":772721569792,"internally_reserved":0,"allocated":22720512,"data_stored":74587653,"data_compressed":739675,"data_compressed_allocated":4517888,"data_compressed_original":64156315,"omap_allocated":12744,"internal_metadata":349687352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-273482169,"num_objects":-97,"num_object_clones":-25,"num_object_copies":-194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-97,"num_whiteouts":-24,"num_read":1654,"num_read_kb":1408,"num_write":215,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024046"},"pg_stats":[{"pgid":"3.1f","version":"141'404","reported_seq":1828,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.989912+0000","last_change":"2024-09-13T09:19:43.990193+0000","last_active":"2024-09-13T09:19:43.989912+0000","last_peered":"2024-09-13T09:19:43.989912+0000","last_clean":"2024-09-13T09:19:43.989912+0000","last_became_active":"2024-09-13T09:17:40.791187+0000","last_became_peered":"2024-09-13T09:17:40.791187+0000","last_unstale":"2024-09-13T09:19:43.989912+0000","last_undegraded":"2024-09-13T09:19:43.989912+0000","last_fullsized":"2024-09-13T09:19:43.989912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:13.280600+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.021502245999999999,"stat_sum":{"num_bytes":262160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":2536,"num_read_kb":13963,"num_write":477,"num_write_kb":12323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.1e","version":"140'622","reported_seq":3208,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.971974+0000","last_change":"2024-09-13T09:19:43.972199+0000","last_active":"2024-09-13T09:19:43.971974+0000","last_peered":"2024-09-13T09:19:43.971974+0000","last_clean":"2024-09-13T09:19:43.971974+0000","last_became_active":"2024-09-13T09:19:20.802379+0000","last_became_peered":"2024-09-13T09:19:20.802379+0000","last_unstale":"2024-09-13T09:19:43.971974+0000","last_undegraded":"2024-09-13T09:19:43.971974+0000","last_fullsized":"2024-09-13T09:19:43.971974+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:20.952086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001933078,"stat_sum":{"num_bytes":407724,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":3595,"num_read_kb":10012,"num_write":840,"num_write_kb":4592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":8249616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.1d","version":"141'462","reported_seq":3612,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.995549+0000","last_change":"2024-09-13T09:19:43.996177+0000","last_active":"2024-09-13T09:19:43.995549+0000","last_peered":"2024-09-13T09:19:43.995549+0000","last_clean":"2024-09-13T09:19:43.995549+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:19:43.995549+0000","last_undegraded":"2024-09-13T09:19:43.995549+0000","last_fullsized":"2024-09-13T09:19:43.995549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:20.509991+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015737968000000001,"stat_sum":{"num_bytes":389120,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":6,"num_read":5600,"num_read_kb":19906,"num_write":420,"num_write_kb":19537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.1c","version":"141'724","reported_seq":4259,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:44.144649+0000","last_change":"2024-09-13T09:19:43.985794+0000","last_active":"2024-09-13T09:19:44.144649+0000","last_peered":"2024-09-13T09:19:44.144649+0000","last_clean":"2024-09-13T09:19:44.144649+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:19:44.144649+0000","last_undegraded":"2024-09-13T09:19:44.144649+0000","last_fullsized":"2024-09-13T09:19:44.144649+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:05.002231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069747070000000001,"stat_sum":{"num_bytes":700338,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":7324,"num_read_kb":18214,"num_write":821,"num_write_kb":10020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.1b","version":"137'473","reported_seq":4232,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.970966+0000","last_change":"2024-09-13T09:19:43.971420+0000","last_active":"2024-09-13T09:19:43.970966+0000","last_peered":"2024-09-13T09:19:43.970966+0000","last_clean":"2024-09-13T09:19:43.970966+0000","last_became_active":"2024-09-13T09:17:40.799618+0000","last_became_peered":"2024-09-13T09:17:40.799618+0000","last_unstale":"2024-09-13T09:19:43.970966+0000","last_undegraded":"2024-09-13T09:19:43.970966+0000","last_fullsized":"2024-09-13T09:19:43.970966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:00.757881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028611140000000001,"stat_sum":{"num_bytes":8852961,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":5496,"num_read_kb":19387,"num_write":608,"num_write_kb":9125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":65341,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.1a","version":"138'340","reported_seq":3190,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:44.059511+0000","last_change":"2024-09-13T09:19:43.971866+0000","last_active":"2024-09-13T09:19:44.059511+0000","last_peered":"2024-09-13T09:19:44.059511+0000","last_clean":"2024-09-13T09:19:44.059511+0000","last_became_active":"2024-09-13T09:18:03.973063+0000","last_became_peered":"2024-09-13T09:18:03.973063+0000","last_unstale":"2024-09-13T09:19:44.059511+0000","last_undegraded":"2024-09-13T09:19:44.059511+0000","last_fullsized":"2024-09-13T09:19:44.059511+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:29.911994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00107117,"stat_sum":{"num_bytes":317620,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":3155,"num_read_kb":8919,"num_write":340,"num_write_kb":6710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":889520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.19","version":"141'707","reported_seq":4193,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:44.041912+0000","last_change":"2024-09-13T09:19:44.042169+0000","last_active":"2024-09-13T09:19:44.041912+0000","last_peered":"2024-09-13T09:19:44.041912+0000","last_clean":"2024-09-13T09:19:44.041912+0000","last_became_active":"2024-09-13T09:18:06.975016+0000","last_became_peered":"2024-09-13T09:18:06.975016+0000","last_unstale":"2024-09-13T09:19:44.041912+0000","last_undegraded":"2024-09-13T09:19:44.041912+0000","last_fullsized":"2024-09-13T09:19:44.041912+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:30.489567+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.003129157,"stat_sum":{"num_bytes":6355585,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":5493,"num_read_kb":28565,"num_write":1007,"num_write_kb":11903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":439303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.18","version":"134'209","reported_seq":2500,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.970426+0000","last_change":"2024-09-13T09:19:43.971510+0000","last_active":"2024-09-13T09:19:43.970426+0000","last_peered":"2024-09-13T09:19:43.970426+0000","last_clean":"2024-09-13T09:19:43.970426+0000","last_became_active":"2024-09-13T09:17:40.797607+0000","last_became_peered":"2024-09-13T09:17:40.797607+0000","last_unstale":"2024-09-13T09:19:43.970426+0000","last_undegraded":"2024-09-13T09:19:43.970426+0000","last_fullsized":"2024-09-13T09:19:43.970426+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:09.025976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018574290000000001,"stat_sum":{"num_bytes":12200457,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":2065,"num_read_kb":17454,"num_write":175,"num_write_kb":10975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":42856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.17","version":"140'410","reported_seq":2128,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.971389+0000","last_change":"2024-09-13T09:19:43.971789+0000","last_active":"2024-09-13T09:19:43.971389+0000","last_peered":"2024-09-13T09:19:43.971389+0000","last_clean":"2024-09-13T09:19:43.971389+0000","last_became_active":"2024-09-13T09:17:43.809533+0000","last_became_peered":"2024-09-13T09:17:43.809533+0000","last_unstale":"2024-09-13T09:19:43.971389+0000","last_undegraded":"2024-09-13T09:19:43.971389+0000","last_fullsized":"2024-09-13T09:19:43.971389+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:20.389195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003190862,"stat_sum":{"num_bytes":2498244,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":2844,"num_read_kb":24302,"num_write":360,"num_write_kb":12949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":16384,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.16","version":"141'265","reported_seq":2224,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:44.124239+0000","last_change":"2024-09-13T09:19:44.002124+0000","last_active":"2024-09-13T09:19:44.124239+0000","last_peered":"2024-09-13T09:19:44.124239+0000","last_clean":"2024-09-13T09:19:44.124239+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:19:44.124239+0000","last_undegraded":"2024-09-13T09:19:44.124239+0000","last_fullsized":"2024-09-13T09:19:44.124239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:29.849557+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.021103236000000001,"stat_sum":{"num_bytes":3917592,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":1901,"num_read_kb":15086,"num_write":242,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.15","version":"139'360","reported_seq":2906,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.981069+0000","last_change":"2024-09-13T09:19:43.982215+0000","last_active":"2024-09-13T09:19:43.981069+0000","last_peered":"2024-09-13T09:19:43.981069+0000","last_clean":"2024-09-13T09:19:43.981069+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:19:43.981069+0000","last_undegraded":"2024-09-13T09:19:43.981069+0000","last_fullsized":"2024-09-13T09:19:43.981069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:10.717628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003879726,"stat_sum":{"num_bytes":2143014,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":3481,"num_read_kb":18288,"num_write":409,"num_write_kb":15933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.14","version":"141'577","reported_seq":3022,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.984693+0000","last_change":"2024-09-13T09:19:43.985676+0000","last_active":"2024-09-13T09:19:43.984693+0000","last_peered":"2024-09-13T09:19:43.984693+0000","last_clean":"2024-09-13T09:19:43.984693+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:19:43.984693+0000","last_undegraded":"2024-09-13T09:19:43.984693+0000","last_fullsized":"2024-09-13T09:19:43.984693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:42.341412+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014734184000000001,"stat_sum":{"num_bytes":2662664,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":3468,"num_read_kb":17628,"num_write":761,"num_write_kb":18883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-13T09:19:42.969750+0000","last_change":"2024-09-13T09:19:17.796958+0000","last_active":"2024-09-13T09:19:42.969750+0000","last_peered":"2024-09-13T09:19:42.969750+0000","last_clean":"2024-09-13T09:19:42.969750+0000","last_became_active":"2024-09-13T09:19:17.796177+0000","last_became_peered":"2024-09-13T09:19:17.796177+0000","last_unstale":"2024-09-13T09:19:42.969750+0000","last_undegraded":"2024-09-13T09:19:42.969750+0000","last_fullsized":"2024-09-13T09:19:42.969750+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:59.576027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"140'413","reported_seq":1661,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.970353+0000","last_change":"2024-09-13T09:19:43.970689+0000","last_active":"2024-09-13T09:19:43.970353+0000","last_peered":"2024-09-13T09:19:43.970353+0000","last_clean":"2024-09-13T09:19:43.970353+0000","last_became_active":"2024-09-13T09:19:14.748293+0000","last_became_peered":"2024-09-13T09:19:14.748293+0000","last_unstale":"2024-09-13T09:19:43.970353+0000","last_undegraded":"2024-09-13T09:19:43.970353+0000","last_fullsized":"2024-09-13T09:19:43.970353+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:05.560046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002197708,"stat_sum":{"num_bytes":2668215,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":2376,"num_read_kb":17218,"num_write":336,"num_write_kb":10501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":7471802,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":121,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-13T09:19:42.989379+0000","last_change":"2024-09-13T09:19:17.770210+0000","last_active":"2024-09-13T09:19:42.989379+0000","last_peered":"2024-09-13T09:19:42.989379+0000","last_clean":"2024-09-13T09:19:42.989379+0000","last_became_active":"2024-09-13T09:19:17.769570+0000","last_became_peered":"2024-09-13T09:19:17.769570+0000","last_unstale":"2024-09-13T09:19:42.989379+0000","last_undegraded":"2024-09-13T09:19:42.989379+0000","last_fullsized":"2024-09-13T09:19:42.989379+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:37.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"140'552","reported_seq":2952,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.982498+0000","last_change":"2024-09-13T09:19:43.982882+0000","last_active":"2024-09-13T09:19:43.982498+0000","last_peered":"2024-09-13T09:19:43.982498+0000","last_clean":"2024-09-13T09:19:43.982498+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:19:43.982498+0000","last_undegraded":"2024-09-13T09:19:43.982498+0000","last_fullsized":"2024-09-13T09:19:43.982498+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:51.225828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01236434,"stat_sum":{"num_bytes":2187264,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":6,"num_read":4402,"num_read_kb":8559,"num_write":724,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":152,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-13T09:19:42.972247+0000","last_change":"2024-09-13T09:17:40.794662+0000","last_active":"2024-09-13T09:19:42.972247+0000","last_peered":"2024-09-13T09:19:42.972247+0000","last_clean":"2024-09-13T09:19:42.972247+0000","last_became_active":"2024-09-13T09:17:40.793882+0000","last_became_peered":"2024-09-13T09:17:40.793882+0000","last_unstale":"2024-09-13T09:19:42.972247+0000","last_undegraded":"2024-09-13T09:19:42.972247+0000","last_fullsized":"2024-09-13T09:19:42.972247+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:22.922753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"140'369","reported_seq":1565,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:44.039673+0000","last_change":"2024-09-13T09:19:44.040125+0000","last_active":"2024-09-13T09:19:44.039673+0000","last_peered":"2024-09-13T09:19:44.039673+0000","last_clean":"2024-09-13T09:19:44.039673+0000","last_became_active":"2024-09-13T09:18:03.973314+0000","last_became_peered":"2024-09-13T09:18:03.973314+0000","last_unstale":"2024-09-13T09:19:44.039673+0000","last_undegraded":"2024-09-13T09:19:44.039673+0000","last_fullsized":"2024-09-13T09:19:44.039673+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.256457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090454900000000004,"stat_sum":{"num_bytes":5121996,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":11,"num_read":1399,"num_read_kb":26203,"num_write":285,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":167476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-13T09:19:42.989144+0000","last_change":"2024-09-13T09:18:03.968679+0000","last_active":"2024-09-13T09:19:42.989144+0000","last_peered":"2024-09-13T09:19:42.989144+0000","last_clean":"2024-09-13T09:19:42.989144+0000","last_became_active":"2024-09-13T09:18:03.967133+0000","last_became_peered":"2024-09-13T09:18:03.967133+0000","last_unstale":"2024-09-13T09:19:42.989144+0000","last_undegraded":"2024-09-13T09:19:42.989144+0000","last_fullsized":"2024-09-13T09:19:42.989144+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:06.643934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"141'336","reported_seq":2205,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.994883+0000","last_change":"2024-09-13T09:19:43.994883+0000","last_active":"2024-09-13T09:19:43.994883+0000","last_peered":"2024-09-13T09:19:43.994883+0000","last_clean":"2024-09-13T09:19:43.994883+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:19:43.994883+0000","last_undegraded":"2024-09-13T09:19:43.994883+0000","last_fullsized":"2024-09-13T09:19:43.994883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:00.161632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013859698,"stat_sum":{"num_bytes":2422586,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":2135,"num_read_kb":9017,"num_write":352,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.3","version":"141'622","reported_seq":4642,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:44.098601+0000","last_change":"2024-09-13T09:19:43.973000+0000","last_active":"2024-09-13T09:19:44.098601+0000","last_peered":"2024-09-13T09:19:44.098601+0000","last_clean":"2024-09-13T09:19:44.098601+0000","last_became_active":"2024-09-13T09:17:40.800027+0000","last_became_peered":"2024-09-13T09:17:40.800027+0000","last_unstale":"2024-09-13T09:19:44.098601+0000","last_undegraded":"2024-09-13T09:19:44.098601+0000","last_fullsized":"2024-09-13T09:19:44.098601+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:10.182783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002180146,"stat_sum":{"num_bytes":5775141,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":5914,"num_read_kb":14354,"num_write":682,"num_write_kb":6956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":29248,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"2.2","version":"21'2","reported_seq":173,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-13T09:19:42.961011+0000","last_change":"2024-09-13T09:18:03.976890+0000","last_active":"2024-09-13T09:19:42.961011+0000","last_peered":"2024-09-13T09:19:42.961011+0000","last_clean":"2024-09-13T09:19:42.961011+0000","last_became_active":"2024-09-13T09:18:03.970835+0000","last_became_peered":"2024-09-13T09:18:03.970835+0000","last_unstale":"2024-09-13T09:19:42.961011+0000","last_undegraded":"2024-09-13T09:19:42.961011+0000","last_fullsized":"2024-09-13T09:19:42.961011+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:05.189083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"140'554","reported_seq":2620,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:44.039332+0000","last_change":"2024-09-13T09:19:44.039600+0000","last_active":"2024-09-13T09:19:44.039332+0000","last_peered":"2024-09-13T09:19:44.039332+0000","last_clean":"2024-09-13T09:19:44.039332+0000","last_became_active":"2024-09-13T09:18:03.970479+0000","last_became_peered":"2024-09-13T09:18:03.970479+0000","last_unstale":"2024-09-13T09:19:44.039332+0000","last_undegraded":"2024-09-13T09:19:44.039332+0000","last_fullsized":"2024-09-13T09:19:44.039332+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:06.229840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037168100000000001,"stat_sum":{"num_bytes":2171198,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":5533,"num_read_kb":12568,"num_write":630,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1031390,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-13T09:19:42.964379+0000","last_change":"2024-09-13T09:19:28.855977+0000","last_active":"2024-09-13T09:19:42.964379+0000","last_peered":"2024-09-13T09:19:42.964379+0000","last_clean":"2024-09-13T09:19:42.964379+0000","last_became_active":"2024-09-13T09:19:28.855258+0000","last_became_peered":"2024-09-13T09:19:28.855258+0000","last_unstale":"2024-09-13T09:19:42.964379+0000","last_undegraded":"2024-09-13T09:19:42.964379+0000","last_fullsized":"2024-09-13T09:19:42.964379+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:13.467197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"140'468","reported_seq":3382,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.968381+0000","last_change":"2024-09-13T09:19:43.968885+0000","last_active":"2024-09-13T09:19:43.968381+0000","last_peered":"2024-09-13T09:19:43.968381+0000","last_clean":"2024-09-13T09:19:43.968381+0000","last_became_active":"2024-09-13T09:17:40.793065+0000","last_became_peered":"2024-09-13T09:17:40.793065+0000","last_unstale":"2024-09-13T09:19:43.968381+0000","last_undegraded":"2024-09-13T09:19:43.968381+0000","last_fullsized":"2024-09-13T09:19:43.968381+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:38.586460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062545799999999996,"stat_sum":{"num_bytes":4533912,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":4025,"num_read_kb":26901,"num_write":436,"num_write_kb":18254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-13T09:19:42.971630+0000","last_change":"2024-09-13T09:18:03.968508+0000","last_active":"2024-09-13T09:19:42.971630+0000","last_peered":"2024-09-13T09:19:42.971630+0000","last_clean":"2024-09-13T09:19:42.971630+0000","last_became_active":"2024-09-13T09:18:03.968092+0000","last_became_peered":"2024-09-13T09:18:03.968092+0000","last_unstale":"2024-09-13T09:19:42.971630+0000","last_undegraded":"2024-09-13T09:19:42.971630+0000","last_fullsized":"2024-09-13T09:19:42.971630+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:39.649827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"141'771","reported_seq":2722,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.988997+0000","last_change":"2024-09-13T09:19:43.989779+0000","last_active":"2024-09-13T09:19:43.988997+0000","last_peered":"2024-09-13T09:19:43.988997+0000","last_clean":"2024-09-13T09:19:43.988997+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:19:43.988997+0000","last_undegraded":"2024-09-13T09:19:43.988997+0000","last_fullsized":"2024-09-13T09:19:43.988997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:44.094850+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.019316735000000002,"stat_sum":{"num_bytes":2237344,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":10,"num_read":4593,"num_read_kb":13736,"num_write":1063,"num_write_kb":11812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"2.3","version":"19'1","reported_seq":136,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-13T09:19:42.960916+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:19:42.960916+0000","last_peered":"2024-09-13T09:19:42.960916+0000","last_clean":"2024-09-13T09:19:42.960916+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:19:42.960916+0000","last_undegraded":"2024-09-13T09:19:42.960916+0000","last_fullsized":"2024-09-13T09:19:42.960916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":327,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-13T09:19:42.971744+0000","last_change":"2024-09-13T09:17:43.803525+0000","last_active":"2024-09-13T09:19:42.971744+0000","last_peered":"2024-09-13T09:19:42.971744+0000","last_clean":"2024-09-13T09:19:42.971744+0000","last_became_active":"2024-09-13T09:17:43.802603+0000","last_became_peered":"2024-09-13T09:17:43.802603+0000","last_unstale":"2024-09-13T09:19:42.971744+0000","last_undegraded":"2024-09-13T09:19:42.971744+0000","last_fullsized":"2024-09-13T09:19:42.971744+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:07.561959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"138'645","reported_seq":2665,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.970584+0000","last_change":"2024-09-13T09:19:43.971097+0000","last_active":"2024-09-13T09:19:43.970584+0000","last_peered":"2024-09-13T09:19:43.970584+0000","last_clean":"2024-09-13T09:19:43.970584+0000","last_became_active":"2024-09-13T09:18:03.974569+0000","last_became_peered":"2024-09-13T09:18:03.974569+0000","last_unstale":"2024-09-13T09:19:43.970584+0000","last_undegraded":"2024-09-13T09:19:43.970584+0000","last_fullsized":"2024-09-13T09:19:43.970584+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:43.479751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012697050000000001,"stat_sum":{"num_bytes":1422591,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":4123,"num_read_kb":16862,"num_write":733,"num_write_kb":13412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":741496,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.9","version":"137'529","reported_seq":2747,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.971068+0000","last_change":"2024-09-13T09:19:43.971826+0000","last_active":"2024-09-13T09:19:43.971068+0000","last_peered":"2024-09-13T09:19:43.971068+0000","last_clean":"2024-09-13T09:19:43.971068+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:19:43.971068+0000","last_undegraded":"2024-09-13T09:19:43.971068+0000","last_fullsized":"2024-09-13T09:19:43.971068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:06.071033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001119133,"stat_sum":{"num_bytes":30,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":4897,"num_read_kb":11578,"num_write":614,"num_write_kb":5754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.a","version":"140'494","reported_seq":3820,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:44.001822+0000","last_change":"2024-09-13T09:19:44.002548+0000","last_active":"2024-09-13T09:19:44.001822+0000","last_peered":"2024-09-13T09:19:44.001822+0000","last_clean":"2024-09-13T09:19:44.001822+0000","last_became_active":"2024-09-13T09:19:34.891064+0000","last_became_peered":"2024-09-13T09:19:34.891064+0000","last_unstale":"2024-09-13T09:19:44.001822+0000","last_undegraded":"2024-09-13T09:19:44.001822+0000","last_fullsized":"2024-09-13T09:19:44.001822+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:11.410988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010361610000000001,"stat_sum":{"num_bytes":1675264,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":5,"num_read":3945,"num_read_kb":15852,"num_write":460,"num_write_kb":7750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":4249683,"num_keys_recovered":143,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.b","version":"141'421","reported_seq":1777,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.982436+0000","last_change":"2024-09-13T09:19:43.982436+0000","last_active":"2024-09-13T09:19:43.982436+0000","last_peered":"2024-09-13T09:19:43.982436+0000","last_clean":"2024-09-13T09:19:43.982436+0000","last_became_active":"2024-09-13T09:17:40.801195+0000","last_became_peered":"2024-09-13T09:17:40.801195+0000","last_unstale":"2024-09-13T09:19:43.982436+0000","last_undegraded":"2024-09-13T09:19:43.982436+0000","last_fullsized":"2024-09-13T09:19:43.982436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:00:07.773428+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012483947,"stat_sum":{"num_bytes":1813711,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":2626,"num_read_kb":8389,"num_write":518,"num_write_kb":11747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.c","version":"138'670","reported_seq":1630,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.996930+0000","last_change":"2024-09-13T09:19:43.996930+0000","last_active":"2024-09-13T09:19:43.996930+0000","last_peered":"2024-09-13T09:19:43.996930+0000","last_clean":"2024-09-13T09:19:43.996930+0000","last_became_active":"2024-09-13T09:17:26.698657+0000","last_became_peered":"2024-09-13T09:17:26.698657+0000","last_unstale":"2024-09-13T09:19:43.996930+0000","last_undegraded":"2024-09-13T09:19:43.996930+0000","last_fullsized":"2024-09-13T09:19:43.996930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:12.388714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015493722999999999,"stat_sum":{"num_bytes":223424,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":2658,"num_read_kb":8274,"num_write":931,"num_write_kb":9153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.d","version":"141'679","reported_seq":3168,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.994728+0000","last_change":"2024-09-13T09:19:43.994728+0000","last_active":"2024-09-13T09:19:43.994728+0000","last_peered":"2024-09-13T09:19:43.994728+0000","last_clean":"2024-09-13T09:19:43.994728+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:19:43.994728+0000","last_undegraded":"2024-09-13T09:19:43.994728+0000","last_fullsized":"2024-09-13T09:19:43.994728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:25.411265+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015919606999999999,"stat_sum":{"num_bytes":4389092,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":4417,"num_read_kb":31654,"num_write":728,"num_write_kb":30325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.e","version":"139'380","reported_seq":1323,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.984566+0000","last_change":"2024-09-13T09:19:43.984566+0000","last_active":"2024-09-13T09:19:43.984566+0000","last_peered":"2024-09-13T09:19:43.984566+0000","last_clean":"2024-09-13T09:19:43.984566+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:19:43.984566+0000","last_undegraded":"2024-09-13T09:19:43.984566+0000","last_fullsized":"2024-09-13T09:19:43.984566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:31.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005408202,"stat_sum":{"num_bytes":305332,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":1301,"num_read_kb":10607,"num_write":403,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.f","version":"139'973","reported_seq":3992,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.985313+0000","last_change":"2024-09-13T09:19:43.985313+0000","last_active":"2024-09-13T09:19:43.985313+0000","last_peered":"2024-09-13T09:19:43.985313+0000","last_clean":"2024-09-13T09:19:43.985313+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:19:43.985313+0000","last_undegraded":"2024-09-13T09:19:43.985313+0000","last_fullsized":"2024-09-13T09:19:43.985313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":973,"log_dups_size":0,"ondisk_log_size":973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:56.423075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060478629999999997,"stat_sum":{"num_bytes":3860371,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":7123,"num_read_kb":22781,"num_write":1441,"num_write_kb":10311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.10","version":"136'395","reported_seq":2254,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:44.001665+0000","last_change":"2024-09-13T09:19:44.002281+0000","last_active":"2024-09-13T09:19:44.001665+0000","last_peered":"2024-09-13T09:19:44.001665+0000","last_clean":"2024-09-13T09:19:44.001665+0000","last_became_active":"2024-09-13T09:19:20.803495+0000","last_became_peered":"2024-09-13T09:19:20.803495+0000","last_unstale":"2024-09-13T09:19:44.001665+0000","last_undegraded":"2024-09-13T09:19:44.001665+0000","last_fullsized":"2024-09-13T09:19:44.001665+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:18.288761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081310300000000002,"stat_sum":{"num_bytes":356352,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":7,"num_read":2117,"num_read_kb":10151,"num_write":495,"num_write_kb":12845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5102507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.11","version":"136'546","reported_seq":2955,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.982547+0000","last_change":"2024-09-13T09:19:43.983560+0000","last_active":"2024-09-13T09:19:43.982547+0000","last_peered":"2024-09-13T09:19:43.982547+0000","last_clean":"2024-09-13T09:19:43.982547+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:19:43.982547+0000","last_undegraded":"2024-09-13T09:19:43.982547+0000","last_fullsized":"2024-09-13T09:19:43.982547+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:47.572839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052067340000000002,"stat_sum":{"num_bytes":3830682,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":4916,"num_read_kb":14560,"num_write":712,"num_write_kb":15391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.12","version":"139'431","reported_seq":4878,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.970279+0000","last_change":"2024-09-13T09:19:43.971162+0000","last_active":"2024-09-13T09:19:43.970279+0000","last_peered":"2024-09-13T09:19:43.970279+0000","last_clean":"2024-09-13T09:19:43.970279+0000","last_became_active":"2024-09-13T09:17:40.789499+0000","last_became_peered":"2024-09-13T09:17:40.789499+0000","last_unstale":"2024-09-13T09:19:43.970279+0000","last_undegraded":"2024-09-13T09:19:43.970279+0000","last_fullsized":"2024-09-13T09:19:43.970279+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:13.395268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027974889999999998,"stat_sum":{"num_bytes":422011,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":7,"num_read":5803,"num_read_kb":44996,"num_write":397,"num_write_kb":17054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"5"}]},{"pgid":"3.13","version":"138'411","reported_seq":3556,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-13T09:19:43.980908+0000","last_change":"2024-09-13T09:19:43.982142+0000","last_active":"2024-09-13T09:19:43.980908+0000","last_peered":"2024-09-13T09:19:43.980908+0000","last_clean":"2024-09-13T09:19:43.980908+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:19:43.980908+0000","last_undegraded":"2024-09-13T09:19:43.980908+0000","last_fullsized":"2024-09-13T09:19:43.980908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:07.893848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038421689999999999,"stat_sum":{"num_bytes":1910515,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":4116,"num_read_kb":18422,"num_write":404,"num_write_kb":9869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":88034510,"num_objects":386,"num_object_clones":60,"num_object_copies":772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":386,"num_whiteouts":203,"num_read":125377,"num_read_kb":554406,"num_write":18804,"num_write_kb":379071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1378,"num_bytes_recovered":28693566,"num_keys_recovered":250,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3891200,"data_stored":5807258,"data_compressed":183910,"data_compressed_allocated":405504,"data_compressed_original":2855579,"omap_allocated":0,"internal_metadata":0},"log_size":16212,"ondisk_log_size":16212,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":30095,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56436,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315404,"statfs":{"total":96636764160,"available":96578973696,"internally_reserved":0,"allocated":3252224,"data_stored":14332638,"data_compressed":106350,"data_compressed_allocated":864256,"data_compressed_original":12947456,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":0.94799999999999995,"15min":0.88900000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":11.683,"5min":11.683,"15min":11.683},"last":1.034},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.03,"15min":0.98199999999999998},"min":{"1min":0.48099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":11.843999999999999,"5min":11.843999999999999,"15min":11.843999999999999},"last":0.91200000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.86299999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.222,"5min":1.286,"15min":1.286},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.95199999999999996,"15min":0.98399999999999999},"min":{"1min":0.51200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.089,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":1.109}]},{"osd":2,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80000000000000004,"15min":0.80300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.94899999999999995,"15min":0.95899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.24,"5min":1.385,"15min":1.385},"last":1.091}]},{"osd":3,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.038,"15min":0.97399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":12.012,"5min":12.012,"15min":12.012},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.135,"15min":1.075},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":11.776999999999999,"5min":11.776999999999999,"15min":11.776999999999999},"last":0.82699999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":1.028,"15min":1.0609999999999999},"min":{"1min":0.44400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2969999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.4590000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.93400000000000005}]},{"osd":5,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.073,"15min":1.071},"min":{"1min":0.61199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.121,"15min":1.145},"min":{"1min":0.51900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":3.125,"15min":3.125},"last":0.998}]},{"osd":6,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.1950000000000001,"15min":1.2330000000000001},"min":{"1min":0.59199999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.667,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.109,"15min":1.119},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.498,"5min":1.996,"15min":1.996},"last":1.151}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41000,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330840,"statfs":{"total":96636764160,"available":96594780160,"internally_reserved":0,"allocated":1732608,"data_stored":826809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.79100000000000004,"15min":0.79800000000000004},"min":{"1min":0.36799999999999999,"5min":0.314,"15min":0.314},"max":{"1min":1.1060000000000001,"5min":1.206,"15min":1.206},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.871,"15min":0.88200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.093,"5min":2.222,"15min":2.222},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79800000000000004,"15min":0.80000000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":3.0409999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.96999999999999997,"15min":0.97599999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.24,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.84399999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.97099999999999997,"15min":0.99299999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.427,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.98999999999999999,"15min":1.004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.25,"5min":2.508,"15min":2.508},"last":1.337}]},{"osd":3,"last update":"Fri Sep 13 09:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.997,"15min":1.0029999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.6659999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.526},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.89500000000000002,"15min":0.90900000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0600000000000001,"5min":1.546,"15min":1.546},"last":3.2450000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.96799999999999997,"15min":0.97899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4019999999999999,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.997},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.99399999999999999,"15min":0.995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.51,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.4359999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.89200000000000002,"15min":0.89000000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":3.1320000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0509999999999999,"15min":1.0549999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5629999999999999,"5min":2.286,"15min":2.286},"last":1.8120000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:19:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.58699999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5800000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.611},{"interface":"front","average":{"1min":1.081,"5min":1.0429999999999999,"15min":1.0369999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":3.327}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57352,"kb_used_data":4092,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94314488,"statfs":{"total":96636764160,"available":96578035712,"internally_reserved":0,"allocated":4190208,"data_stored":26003554,"data_compressed":233603,"data_compressed_allocated":1671168,"data_compressed_original":24498176,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":6.3970000000000002,"15min":7.3419999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1739999999999999,"5min":132.32900000000001,"15min":132.32900000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.236,"5min":1.335,"15min":1.335},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.86499999999999999,"15min":0.88500000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96399999999999997,"5min":1.946,"15min":1.946},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.998,"15min":1.0209999999999999},"min":{"1min":0.56899999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1579999999999999,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.85099999999999998,"15min":0.84699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1220000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.88800000000000001,"15min":0.89700000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1719999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.93799999999999994,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2849999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.629},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0009999999999999,"15min":1.006},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4419999999999999,"5min":1.476,"15min":1.476},"last":0.64300000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86899999999999999,"15min":0.86499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3140000000000001,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":6.8070000000000004,"15min":7.7770000000000001},"min":{"1min":0.47699999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.544,"5min":132.23599999999999,"15min":132.23599999999999},"last":0.56100000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.441},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.016,"15min":1.0329999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3160000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.075,"15min":1.075},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.452,"5min":1.655,"15min":1.655},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1279999999999999,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6379999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":0.61699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48420,"kb_used_data":3352,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323420,"statfs":{"total":96636764160,"available":96587182080,"internally_reserved":0,"allocated":3432448,"data_stored":14161256,"data_compressed":163366,"data_compressed_allocated":901120,"data_compressed_original":12607488,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82399999999999995,"15min":0.82999999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.92200000000000004,"15min":0.93999999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1279999999999999,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":1.135}]},{"osd":1,"last update":"Fri Sep 13 09:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.90800000000000003,"15min":0.92400000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.835,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0249999999999999,"5min":1.329,"15min":1.329},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.877,"15min":0.877},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.84399999999999997,"15min":0.83099999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.2050000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.85499999999999998,"15min":0.84899999999999998},"min":{"1min":0.54400000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.002,"15min":0.96999999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.7389999999999999,"5min":4.7389999999999999,"15min":4.7389999999999999},"last":1.0189999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.89000000000000001,"15min":0.88600000000000001},"min":{"1min":0.40899999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.91000000000000003,"15min":0.91400000000000003},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.83299999999999996}]},{"osd":6,"last update":"Fri Sep 13 09:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.026,"15min":1.036},"min":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.454,"5min":1.633,"15min":1.633},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.052,"15min":1.0549999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.052}]},{"osd":7,"last update":"Fri Sep 13 09:19:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0249999999999999,"15min":1.0229999999999999},"min":{"1min":0.60099999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.996},{"interface":"front","average":{"1min":1.079,"5min":1.123,"15min":1.131},"min":{"1min":0.5,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.679,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.3400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55652,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2445312,"data_stored":1738704,"data_compressed":1837,"data_compressed_allocated":36864,"data_compressed_original":328347,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.313,"5min":0.215,"15min":0.215},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.96399999999999997,"5min":1.625,"15min":1.625},"last":0.55800000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.28799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.36899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.425,"5min":1.544,"15min":1.544},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.755,"15min":0.76200000000000001},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.84299999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89200000000000002,"15min":0.90600000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.25,"5min":1.669,"15min":1.669},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.93300000000000005,"15min":0.94199999999999995},"min":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.296,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.1930000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.97199999999999998},"min":{"1min":0.59999999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.5580000000000001,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.92400000000000004,"15min":0.93200000000000005},"min":{"1min":0.63,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3380000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.2629999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.628,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.47,"5min":1.748,"15min":1.748},"last":1.004},{"interface":"front","average":{"1min":1.024,"5min":1.0860000000000001,"15min":1.0960000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5620000000000001,"5min":2.484,"15min":2.484},"last":1.044}]},{"osd":7,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.998,"15min":1.006},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.385,"5min":1.806,"15min":1.806},"last":1.139},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.665,"15min":1.772},"min":{"1min":0.61299999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.3879999999999999,"5min":15.4,"15min":15.4},"last":1.302}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41048,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330792,"statfs":{"total":96636764160,"available":96594731008,"internally_reserved":0,"allocated":1781760,"data_stored":978964,"data_compressed":518,"data_compressed_allocated":4096,"data_compressed_original":114688,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":0.68500000000000005,"15min":0.61499999999999999},"min":{"1min":0.23499999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":9.7509999999999994,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.71499999999999997,"15min":0.70099999999999996},"min":{"1min":0.28100000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.8490000000000002,"5min":2.8490000000000002,"15min":2.8490000000000002},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.92800000000000005,"5min":1.23,"15min":1.23},"last":0.499},{"interface":"front","average":{"1min":0.746,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.738,"5min":1.907,"15min":1.907},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.857,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.59599999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.7709999999999999,"5min":1.782,"15min":1.782},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80000000000000004,"15min":0.80500000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.171,"5min":1.226,"15min":1.226},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.5109999999999999,"5min":1.847,"15min":1.847},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.004,"5min":1.0229999999999999,"15min":1.026},"min":{"1min":0.61299999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.827,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.93200000000000005,"15min":0.93799999999999994},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2709999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.99399999999999999,"15min":0.999},"min":{"1min":0.54000000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3380000000000001,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.94699999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.123,"15min":1.133},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5940000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.81299999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29264,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342576,"statfs":{"total":96636764160,"available":96606797824,"internally_reserved":0,"allocated":2285568,"data_stored":1577292,"data_compressed":58922,"data_compressed_allocated":90112,"data_compressed_original":360448,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.70599999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.429,"5min":1.484,"15min":1.484},"last":0.624}]},{"osd":2,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65500000000000003,"15min":0.67100000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.091,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.034}]},{"osd":3,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.35299999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.212,"5min":1.264,"15min":1.264},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76000000000000001,"15min":0.76700000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.95099999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.92900000000000005,"15min":0.95899999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3029999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.98299999999999998,"15min":1.0129999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.22,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.099}]},{"osd":5,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.0760000000000001,"15min":1.04},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":8.3460000000000001,"5min":8.3460000000000001,"15min":8.3460000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.998,"15min":1.022},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2769999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.133}]},{"osd":6,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.024,"15min":1.0489999999999999},"min":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.339,"5min":1.647,"15min":1.647},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.012,"15min":1.0309999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.389,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.997}]},{"osd":7,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.03,"15min":1.052},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.345,"5min":1.629,"15min":1.629},"last":1.163},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.095,"15min":1.119},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.411,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.0569999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34640,"kb_used_data":3516,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337200,"statfs":{"total":96636764160,"available":96601292800,"internally_reserved":0,"allocated":3600384,"data_stored":14968436,"data_compressed":175079,"data_compressed_allocated":950272,"data_compressed_original":13299712,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.30599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.083},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.61299999999999999,"15min":0.60199999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.123,"5min":1.284,"15min":1.284},"last":0.91100000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71899999999999997,"15min":0.71499999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.129}]},{"osd":3,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.42299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":1.173},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.45900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.2250000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.876,"15min":0.872},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.018},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.038,"15min":0.96799999999999997},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":9.5649999999999995,"5min":9.5649999999999995,"15min":9.5649999999999995},"last":1.048}]},{"osd":5,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.032,"15min":0.95299999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":9.6120000000000001,"5min":9.6120000000000001,"15min":9.6120000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.81599999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.401,"5min":1.476,"15min":1.476},"last":0.84799999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0840000000000001,"15min":1.0920000000000001},"min":{"1min":0.68000000000000005,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4710000000000001,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.371},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.99399999999999999,"15min":0.997},"min":{"1min":0.61099999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.268}]},{"osd":7,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.024,"15min":1.0309999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4430000000000001,"5min":1.617,"15min":1.617},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.1499999999999999,"15min":1.0860000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":9.6440000000000001,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":1.3049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":905216,"data_stored":1395873,"data_compressed":32552,"data_compressed_allocated":77824,"data_compressed_original":643072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":726789,"data_compressed":58839,"data_compressed_allocated":86016,"data_compressed_original":348160,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":182071,"data_compressed":518,"data_compressed_allocated":4096,"data_compressed_original":114688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":700416,"data_stored":881977,"data_compressed":1754,"data_compressed_allocated":32768,"data_compressed_original":316059,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":630784,"data_stored":1096279,"data_compressed":44455,"data_compressed_allocated":90112,"data_compressed_original":618496,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":704723,"data_compressed":31459,"data_compressed_allocated":69632,"data_compressed_original":401408,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":23689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":512000,"data_stored":795857,"data_compressed":14333,"data_compressed_allocated":45056,"data_compressed_original":413696,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:19:45.159 INFO:tasks.thrashosds.thrasher:Setting 3.c to [4, 0, 5, 7] 2024-09-13T09:19:45.159 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.c', '4', '0', '5', '7'] 2024-09-13T09:19:45.160 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.c 4 0 5 7 2024-09-13T09:19:45.181 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:45.181+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:45.257 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:45.257 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:45.258 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:45.258 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:45.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:19:45.320 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:45.316+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:45.384 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:45.380+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:45.389 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:45.389 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:45.389 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:45.403 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:19:45.484 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:45.481+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:45.563 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:19:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 118.02386987, 2024-09-13T09:19:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:19:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:19:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:19:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:19:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:19:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:19:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654302885000007, 2024-09-13T09:19:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:19:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:19:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:19:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654285564000006, 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:19:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654268622999993, 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:19:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654251395000003, 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:19:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:19:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654214973999999, 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:19:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654161512000002, 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:19:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:19:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:19:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:19:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:19:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:19:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654139491999999, 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:19:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:19:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:19:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:19:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654114856999996, 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:19:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:19:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:19:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654096334000002, 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:19:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:19:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:19:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:19:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:19:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:19:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:19:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:19:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:19:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654078463000005, 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:19:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654061132999999, 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:19:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:19:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:19:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:19:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:19:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:19:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654043662999996, 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:19:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:19:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:19:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:19:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654026797, 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:19:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:19:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:19:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:19:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:19:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:19:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.654009009999996, 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.653964700000003, 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:19:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:19:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.420834212000003, 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:19:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:19:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.420817378999999, 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:19:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:19:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.420801021000003, 2024-09-13T09:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:19:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:19:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:19:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:19:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:19:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:19:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:19:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:19:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:19:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:66592 3.7 3:e0817996:::rbd_data.1185b1e141e2.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.916441290999998, 2024-09-13T09:19:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.051168777999999998, 2024-09-13T09:19:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 66592 2024-09-13T09:19:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643787+0000", 2024-09-13T09:19:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643799+0000", 2024-09-13T09:19:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2140000000000001e-05 2024-09-13T09:19:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643827+0000", 2024-09-13T09:19:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8121999999999999e-05 2024-09-13T09:19:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643841+0000", 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3969e-05 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643985+0000", 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014404199999999999 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.644371+0000", 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038588700000000001 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.654317+0000", 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099452050000000004 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694641+0000", 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.040324802999999999 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694682+0000", 2024-09-13T09:19:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0451999999999998e-05 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694828+0000", 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014580499999999999 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694958+0000", 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013029599999999999 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:45.655 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:45.656 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:19:45.657 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:45.653+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:45.758 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:45.757+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:45.791 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:45.791 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:45.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:19:45.894 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:45.893+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:45.960 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:45.956+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:45.976 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:45.976 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:45.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:19:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:set 3.c pg_upmap_items mapping to [4->0,5->7] 2024-09-13T09:19:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:19:46.068 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:46.064+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:46.161 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:46.160+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:46.173 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:46.173 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:46.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.18397301, 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:19:46.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.175 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:46.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:46.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:19:46.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:19:46.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:19:46.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:19:46.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:19:46.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:19:46.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:19:46.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:19:46.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:19:46.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:19:46.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:19:46.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:19:46.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:19:46.178 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.178 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.18392427400001, 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:19:46.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.183 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.183856605, 2024-09-13T09:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:46.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:46.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:19:46.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:19:46.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:19:46.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:19:46.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:19:46.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:19:46.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:19:46.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:19:46.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:19:46.188 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.188 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.188 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.188 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.188 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.18371639599999, 2024-09-13T09:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:19:46.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:19:46.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:19:46.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.193 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.193 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.193 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:46.193 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.18362553, 2024-09-13T09:19:46.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:19:46.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:46.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:19:46.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:19:46.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:19:46.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:19:46.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:19:46.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:19:46.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:19:46.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:19:46.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:19:46.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:19:46.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:19:46.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:19:46.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:19:46.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.198 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.198 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.198 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:46.198 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.18355067, 2024-09-13T09:19:46.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:46.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:19:46.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:19:46.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:19:46.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:19:46.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:19:46.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:19:46.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:19:46.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:19:46.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:19:46.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.203 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.203 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.203 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:46.204 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.183479494, 2024-09-13T09:19:46.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:19:46.204 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.204 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.204 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:19:46.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.204 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:46.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:46.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:19:46.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:19:46.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:19:46.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:19:46.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:19:46.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:19:46.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:19:46.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.208 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.183430441, 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:19:46.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:19:46.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.18336870500001, 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:19:46.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.183319035, 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:19:46.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.18327051599999, 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:19:46.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:19:46.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:19:46.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:19:46.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:19:46.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.227 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.18322212, 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:19:46.231 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.183174833, 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:19:46.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:19:46.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:19:46.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.236 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.18309143099999, 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:46.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:46.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:19:46.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:19:46.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.183017705, 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.241 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:19:46.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.246 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.246 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:46.246 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.18294958200001, 2024-09-13T09:19:46.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:19:46.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:19:46.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:19:46.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:19:46.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:19:46.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:19:46.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.250 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 122.182899115, 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:19:46.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:19:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:19:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:19:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:19:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:19:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:19:46.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:19:46.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.255 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.255 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:78833 3.a 3:5235bc0b:::rbd_data.11853dd604c.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:46.255 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:46.255 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.900629297999998, 2024-09-13T09:19:46.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.35280560599999999, 2024-09-13T09:19:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78833 2024-09-13T09:19:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269309+0000", 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269322+0000", 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3359e-05 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269352+0000", 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0320000000000001e-05 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269367+0000", 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.526e-05 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269434+0000", 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.648e-05 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269446+0000", 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2286999999999999e-05 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.606711+0000", 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.33726426399999998 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.606879+0000", 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016884100000000001 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.609371+0000", 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024920110000000001 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622034+0000", 2024-09-13T09:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012662199000000001 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622042+0000", 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9570000000000002e-06 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622083+0000", 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.197e-05 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622117+0000", 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3013999999999999e-05 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:80090 3.a 3:517b773f:::rbd_object_map.118553778f1e:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.444256002000003, 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99537752499999999, 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:46.262 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80090 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725681+0000", 2024-09-13T09:19:46.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-13T09:19:46.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725699+0000", 2024-09-13T09:19:46.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7683999999999999e-05 2024-09-13T09:19:46.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725746+0000", 2024-09-13T09:19:46.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7196e-05 2024-09-13T09:19:46.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725769+0000", 2024-09-13T09:19:46.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3383e-05 2024-09-13T09:19:46.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725956+0000", 2024-09-13T09:19:46.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000187049 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725985+0000", 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8472000000000001e-05 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.718483+0000", 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99249815200000002 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.718851+0000", 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036844299999999998 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.719201+0000", 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034928699999999999 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720843+0000", 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016425249999999999 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720882+0000", 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9045999999999999e-05 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720985+0000", 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103163 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.721062+0000", 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6402000000000005e-05 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.268 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:81122 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.269415589000001, 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.920409343, 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 81122 2024-09-13T09:19:46.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.270 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:46.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:46.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900523+0000", 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900536+0000", 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3200000000000001e-05 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900565+0000", 2024-09-13T09:19:46.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9703e-05 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900588+0000", 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2096e-05 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900698+0000", 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110029 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:46.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900716+0000", 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8070000000000001e-05 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:17.766722+0000", 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.86600628099999999 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:17.766770+0000", 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8486000000000003e-05 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.817359+0000", 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.050588836 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.817852+0000", 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00049274599999999996 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.818512+0000", 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00066035099999999999 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820216+0000", 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017038680000000001 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820601+0000", 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038506599999999999 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820746+0000", 2024-09-13T09:19:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014458900000000001 2024-09-13T09:19:46.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820934+0000", 2024-09-13T09:19:46.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188102 2024-09-13T09:19:46.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.276 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.276 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.276 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:46.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:19:46.278 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:46.272+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:46.380 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:46.372+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:46.406 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:46.406 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:46.407 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:46.407 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:46.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:19:46.481 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:46.476+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:46.540 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:46.540 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:46.541 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:46.541 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:46.541 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:46.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:19:46.602 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:46.601+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:46.683 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:46.680+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:46.686 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:46.686 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:46.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 137.99806422200001, 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.21372158699999999, 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10 2024-09-13T09:19:46.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685363+0000", 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685372+0000", 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4590000000000004e-06 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685394+0000", 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1616000000000001e-05 2024-09-13T09:19:46.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685407+0000", 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3675e-05 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685484+0000", 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7065999999999997e-05 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:19:46.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685493+0000", 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8070000000000006e-06 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690770+0000", 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052763869999999996 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690940+0000", 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017001900000000001 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.691238+0000", 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029794499999999998 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.692057+0000", 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081943000000000001 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.898933+0000", 2024-09-13T09:19:46.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.20687588200000001 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899026+0000", 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2777999999999998e-05 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899086+0000", 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0188e-05 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.694 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:14590 3.1d 3:be60f898:::rbd_data.11853ecd5f5f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13672152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:19:46.694 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:19:46.694 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 119.14221578199999, 2024-09-13T09:19:46.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052557382, 2024-09-13T09:19:46.694 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:46.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 14590 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541207+0000", 2024-09-13T09:19:46.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-13T09:19:46.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562477+0000", 2024-09-13T09:19:46.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021270155999999998 2024-09-13T09:19:46.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562539+0000", 2024-09-13T09:19:46.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1851000000000001e-05 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562565+0000", 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6194e-05 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562702+0000", 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013731299999999999 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.563071+0000", 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036892899999999999 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.565167+0000", 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020961959999999998 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.585261+0000", 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020093443999999999 2024-09-13T09:19:46.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593596+0000", 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083352849999999996 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593696+0000", 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100021 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593770+0000", 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3889999999999999e-05 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:14584 3.2 e39/22 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head v 39'64, mlcod=38'62)", 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 119.140230136, 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051546091000000002, 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:46.700 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:19:46.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:19:46.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543197+0000", 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588691+0000", 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045493475999999998 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588711+0000", 2024-09-13T09:19:46.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0485000000000001e-05 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588721+0000", 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.499e-06 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588766+0000", 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5228000000000001e-05 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588788+0000", 2024-09-13T09:19:46.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2503000000000001e-05 2024-09-13T09:19:46.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:46.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594671+0000", 2024-09-13T09:19:46.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058822809999999996 2024-09-13T09:19:46.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594744+0000", 2024-09-13T09:19:46.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3776000000000007e-05 2024-09-13T09:19:46.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:22580 3.1c 3:39876c9f:::rbd_data.1185d6f8ec87.000000000000000f:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 100.830977346, 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050372896, 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22580 2024-09-13T09:19:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:19:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:19:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852449+0000", 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852462+0000", 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3756e-05 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852495+0000", 2024-09-13T09:19:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2435999999999999e-05 2024-09-13T09:19:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852516+0000", 2024-09-13T09:19:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1347000000000001e-05 2024-09-13T09:19:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852644+0000", 2024-09-13T09:19:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012777200000000001 2024-09-13T09:19:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852784+0000", 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013980500000000001 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.860936+0000", 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081526859999999993 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:46.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902522+0000", 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041585203000000001 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902560+0000", 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8603000000000003e-05 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902708+0000", 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014740900000000001 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902824+0000", 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116519 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54987 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6447104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.441121089000006, 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041604369000000002, 2024-09-13T09:19:46.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54987 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242309+0000", 2024-09-13T09:19:46.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1160000000000002e-06 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242317+0000", 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9799999999999998e-06 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242323+0000", 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6439999999999997e-06 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283711+0000", 2024-09-13T09:19:46.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388085999999998 2024-09-13T09:19:46.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283766+0000", 2024-09-13T09:19:46.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5112999999999999e-05 2024-09-13T09:19:46.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283912+0000", 2024-09-13T09:19:46.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014575300000000001 2024-09-13T09:19:46.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54988 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6545408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.441101269000001, 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041865404000000002, 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54988 2024-09-13T09:19:46.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.717 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:19:46.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:19:46.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242329+0000", 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242335+0000", 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242341+0000", 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9440000000000002e-06 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283952+0000", 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041610981999999998 2024-09-13T09:19:46.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284076+0000", 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124356 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284193+0000", 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116345 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54989 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6643712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.441083155000001, 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042123657000000002, 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54989 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242348+0000", 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.655e-06 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242354+0000", 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2269999999999998e-06 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242360+0000", 2024-09-13T09:19:46.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6170000000000003e-06 2024-09-13T09:19:46.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284248+0000", 2024-09-13T09:19:46.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041887830000000001 2024-09-13T09:19:46.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284340+0000", 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1358000000000002e-05 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284469+0000", 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012932300000000001 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54990 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6742016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.725 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.441049864999997, 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042278333000000001, 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54990 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242378+0000", 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242381+0000", 2024-09-13T09:19:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0680000000000001e-06 2024-09-13T09:19:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242392+0000", 2024-09-13T09:19:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0762e-05 2024-09-13T09:19:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242398+0000", 2024-09-13T09:19:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1040000000000003e-06 2024-09-13T09:19:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284506+0000", 2024-09-13T09:19:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042107983000000002 2024-09-13T09:19:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284564+0000", 2024-09-13T09:19:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.821e-05 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284657+0000", 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2588000000000002e-05 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54991 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6840320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:19:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.441025073999995, 2024-09-13T09:19:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042489459, 2024-09-13T09:19:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54991 2024-09-13T09:19:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242406+0000", 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242412+0000", 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5919999999999997e-06 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242425+0000", 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2713e-05 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284694+0000", 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042269558999999998 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284784+0000", 2024-09-13T09:19:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9999000000000003e-05 2024-09-13T09:19:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284893+0000", 2024-09-13T09:19:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108422 2024-09-13T09:19:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54992 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6938624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:19:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.440998703000005, 2024-09-13T09:19:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042636897999999999, 2024-09-13T09:19:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54992 2024-09-13T09:19:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242429+0000", 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242432+0000", 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7599999999999998e-06 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242439+0000", 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5760000000000002e-06 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242448+0000", 2024-09-13T09:19:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-13T09:19:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284935+0000", 2024-09-13T09:19:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042486949000000003 2024-09-13T09:19:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284991+0000", 2024-09-13T09:19:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6471000000000002e-05 2024-09-13T09:19:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285067+0000", 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5543000000000005e-05 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54993 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7036928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.440975825999999, 2024-09-13T09:19:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042867959999999997, 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54993 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242452+0000", 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242455+0000", 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4360000000000001e-06 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242463+0000", 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8220000000000007e-06 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242468+0000", 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9479999999999999e-06 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285107+0000", 2024-09-13T09:19:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042639415999999999 2024-09-13T09:19:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285196+0000", 2024-09-13T09:19:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8721000000000007e-05 2024-09-13T09:19:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285321+0000", 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012495 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54994 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7135232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.440948661999997, 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043009622999999997, 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54994 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:19:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:19:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242482+0000", 2024-09-13T09:19:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:19:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242486+0000", 2024-09-13T09:19:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1049999999999997e-06 2024-09-13T09:19:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242489+0000", 2024-09-13T09:19:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.004e-06 2024-09-13T09:19:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285354+0000", 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.04286479 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285413+0000", 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9778000000000003e-05 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285489+0000", 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6141000000000002e-05 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54995 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7233536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.440937290999997, 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043166641999999998, 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54995 2024-09-13T09:19:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:19:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:19:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242492+0000", 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.437e-06 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242496+0000", 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242498+0000", 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5950000000000001e-06 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285528+0000", 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043029755000000003 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285585+0000", 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285658+0000", 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2713999999999995e-05 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54996 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7331840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:19:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.440927975999998, 2024-09-13T09:19:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043333782000000001, 2024-09-13T09:19:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54996 2024-09-13T09:19:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242500+0000", 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242502+0000", 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3599999999999999e-06 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242505+0000", 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1439999999999999e-06 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242507+0000", 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.605e-06 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285698+0000", 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043190911999999998 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285754+0000", 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5541999999999998e-05 2024-09-13T09:19:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285834+0000", 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0426000000000006e-05 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54997 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.440918886999995, 2024-09-13T09:19:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043496174999999998, 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54997 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242509+0000", 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242511+0000", 2024-09-13T09:19:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.313e-06 2024-09-13T09:19:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242514+0000", 2024-09-13T09:19:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:19:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242516+0000", 2024-09-13T09:19:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3999999999999999e-06 2024-09-13T09:19:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285875+0000", 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043358578000000002 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285929+0000", 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4602999999999999e-05 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286006+0000", 2024-09-13T09:19:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6429999999999998e-05 2024-09-13T09:19:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54998 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7528448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:19:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.440826275000006, 2024-09-13T09:19:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043666469999999999, 2024-09-13T09:19:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54998 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242518+0000", 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999161 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242606+0000", 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7607000000000003e-05 2024-09-13T09:19:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242617+0000", 2024-09-13T09:19:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0882e-05 2024-09-13T09:19:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242624+0000", 2024-09-13T09:19:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9009999999999997e-06 2024-09-13T09:19:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286046+0000", 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043422308 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286106+0000", 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0285000000000003e-05 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286269+0000", 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000162578 2024-09-13T09:19:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:55006 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8314880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:19:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.437437607000007, 2024-09-13T09:19:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041461771000000001, 2024-09-13T09:19:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 55006 2024-09-13T09:19:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:19:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:19:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:19:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245993+0000", 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2869999999999999e-06 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245999+0000", 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0159999999999997e-06 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.246004+0000", 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:19:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287345+0000", 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041341217 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287398+0000", 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2840000000000002e-05 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287453+0000", 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4746000000000001e-05 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:68662 3.1d 3:be8be2fa:::rbd_data.118556027ff4.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.526382166000005, 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.044923689000000003, 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 68662 2024-09-13T09:19:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:19:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157045+0000", 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157052+0000", 2024-09-13T09:19:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8750000000000002e-06 2024-09-13T09:19:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157066+0000", 2024-09-13T09:19:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3815000000000001e-05 2024-09-13T09:19:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157073+0000", 2024-09-13T09:19:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.64e-06 2024-09-13T09:19:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157145+0000", 2024-09-13T09:19:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2517999999999999e-05 2024-09-13T09:19:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157267+0000", 2024-09-13T09:19:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012185599999999999 2024-09-13T09:19:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.164012+0000", 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0067447660000000001 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201805+0000", 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037793528999999999 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201881+0000", 2024-09-13T09:19:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5678000000000003e-05 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201933+0000", 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2346999999999999e-05 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201970+0000", 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6532000000000002e-05 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:78869 3.5 3:a7c01924:::rbd_data.11853dd604c.000000000000000b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:59.731157+0000", 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.952271453999998, 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062116486999999998, 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78869 2024-09-13T09:19:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731157+0000", 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731157+0000", 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731155+0000", 2024-09-13T09:19:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:19:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731171+0000", 2024-09-13T09:19:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5543e-05 2024-09-13T09:19:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731202+0000", 2024-09-13T09:19:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1229999999999997e-05 2024-09-13T09:19:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731217+0000", 2024-09-13T09:19:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5173e-05 2024-09-13T09:19:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731336+0000", 2024-09-13T09:19:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119026 2024-09-13T09:19:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731566+0000", 2024-09-13T09:19:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022948399999999999 2024-09-13T09:19:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-13T09:19:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.742507+0000", 2024-09-13T09:19:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010941383000000001 2024-09-13T09:19:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.793038+0000", 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050530670999999999 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.793060+0000", 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2002999999999999e-05 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.793155+0000", 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4716000000000006e-05 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.793274+0000", 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011882799999999999 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:79042 3.1c 3:3f2730ec:::rbd_data.11853dd604c.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-13T09:19:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:00.228566+0000", 2024-09-13T09:19:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.454862319999997, 2024-09-13T09:19:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066070161000000002, 2024-09-13T09:19:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 79042 2024-09-13T09:19:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228566+0000", 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228566+0000", 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228564+0000", 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228575+0000", 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228605+0000", 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0426e-05 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228620+0000", 2024-09-13T09:19:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4919999999999999e-05 2024-09-13T09:19:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228720+0000", 2024-09-13T09:19:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9889000000000005e-05 2024-09-13T09:19:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228845+0000", 2024-09-13T09:19:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012490600000000001 2024-09-13T09:19:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.241066+0000", 2024-09-13T09:19:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012220795 2024-09-13T09:19:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.294366+0000", 2024-09-13T09:19:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053300301000000001 2024-09-13T09:19:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.294481+0000", 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114629 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.294571+0000", 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0093000000000006e-05 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.294636+0000", 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5605999999999995e-05 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:46.795 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:46.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:19:46.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:19:46.798 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:46.793+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:46.898 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:46.891+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:46.952 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:46.969 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:46.969 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:46.974 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:46.974 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:46.990 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:19:46.990 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:46.999 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:46.991+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:47.008 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:47.008 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:47.024 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:47.024 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:47.041 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:47.058 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:47.076 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:47.118 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:47.118 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:47.133 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:19:47.136 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:47.131+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:47.201 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:47.195+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:47.257 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:47.258 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:47.273 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:19:47.317 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:47.317+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:47.394 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:47.394 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:47.403 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:47.399+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:47.408 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:19:47.503 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:47.495+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:47.578 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:47.579 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:47.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:19:47.637 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:47.637+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:47.705 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:47.705+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:47.722 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:47.722 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:47.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:19:47.806 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:47.805+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:47.872 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:47.872 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:47.887 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:19:47.907 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:47.905+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:48.007 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:48.005+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:48.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:19:48.028 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:48.028 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:48.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:19:48.132 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:48.127+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:48.209 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:48.203+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:48.229 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:48.229 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:48.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:19:48.332 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:48.327+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:48.406 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:48.406 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:48.411 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:48.409+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:48.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:19:48.512 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:48.507+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:48.590 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:48.591 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:48.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:19:48.613 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:48.613+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:48.714 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:48.713+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:19:48.793 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:48.793 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:48.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:19:48.856 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:48.851+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:48.915 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:48.913+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:48.934 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:48.934 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:48.952 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:19:49.040 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:49.035+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:49.093 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:19:49.128 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:49.128 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:49.128 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:49.123+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:49.143 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:19:49.217 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:49.213+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:49.250 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:49.266 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:49.267 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:49.283 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:49.283 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:49.299 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:49.307 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33548 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.315299082999999, 2024-09-13T09:19:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025760904000000001, 2024-09-13T09:19:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33548 2024-09-13T09:19:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990122+0000", 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990124+0000", 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.075e-06 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990130+0000", 2024-09-13T09:19:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.990135+0000", 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007826+0000", 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017691018999999999 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.007896+0000", 2024-09-13T09:19:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0059000000000001e-05 2024-09-13T09:19:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.015883+0000", 2024-09-13T09:19:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079875269999999995 2024-09-13T09:19:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33751 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1593344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311879429000001, 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023779101, 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33751 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993542+0000", 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993548+0000", 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.776e-06 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993555+0000", 2024-09-13T09:19:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993560+0000", 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017188+0000", 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023627984000000001 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017271+0000", 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2798000000000002e-05 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017321+0000", 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0557999999999999e-05 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33752 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311857309000004, 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023925173000000001, 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33752 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993564+0000", 2024-09-13T09:19:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993566+0000", 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993573+0000", 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.759e-06 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993578+0000", 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8339999999999998e-06 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017349+0000", 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023770959000000001 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017423+0000", 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3652000000000006e-05 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017489+0000", 2024-09-13T09:19:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6748999999999994e-05 2024-09-13T09:19:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33753 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1650688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311838827000003, 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024066349000000001, 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33753 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993583+0000", 2024-09-13T09:19:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993582+0000", 2024-09-13T09:19:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993585+0000", 2024-09-13T09:19:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:19:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993591+0000", 2024-09-13T09:19:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:19:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993595+0000", 2024-09-13T09:19:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:19:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017515+0000", 2024-09-13T09:19:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023920117000000001 2024-09-13T09:19:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017583+0000", 2024-09-13T09:19:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8485999999999994e-05 2024-09-13T09:19:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017649+0000", 2024-09-13T09:19:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:19:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33754 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311822532999997, 2024-09-13T09:19:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02420106, 2024-09-13T09:19:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33754 2024-09-13T09:19:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993599+0000", 2024-09-13T09:19:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993601+0000", 2024-09-13T09:19:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:19:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993607+0000", 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9669999999999998e-06 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993611+0000", 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017669+0000", 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024057683999999999 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017742+0000", 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3397000000000002e-05 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017800+0000", 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7711999999999999e-05 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33755 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1708032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311806114000007, 2024-09-13T09:19:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024346923999999999, 2024-09-13T09:19:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33755 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993615+0000", 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993617+0000", 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993624+0000", 2024-09-13T09:19:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0469999999999999e-06 2024-09-13T09:19:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993628+0000", 2024-09-13T09:19:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:19:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017830+0000", 2024-09-13T09:19:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024201879999999999 2024-09-13T09:19:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017897+0000", 2024-09-13T09:19:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7096999999999998e-05 2024-09-13T09:19:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017962+0000", 2024-09-13T09:19:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5455000000000003e-05 2024-09-13T09:19:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33756 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311789387999994, 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024485512000000001, 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33756 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993632+0000", 2024-09-13T09:19:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993635+0000", 2024-09-13T09:19:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:19:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993641+0000", 2024-09-13T09:19:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:19:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993645+0000", 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2520000000000001e-06 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.017979+0000", 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024333950999999999 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018045+0000", 2024-09-13T09:19:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5843e-05 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018118+0000", 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2417999999999997e-05 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33757 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1765376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311771989999997, 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024605578999999999, 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33757 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993650+0000", 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993649+0000", 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993651+0000", 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.841e-06 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993657+0000", 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993662+0000", 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018139+0000", 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024476748 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018196+0000", 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7188000000000002e-05 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018255+0000", 2024-09-13T09:19:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9045999999999997e-05 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33758 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311755356999996, 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024736898, 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33758 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993666+0000", 2024-09-13T09:19:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993668+0000", 2024-09-13T09:19:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:19:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993674+0000", 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993679+0000", 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.515e-06 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018284+0000", 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024604816000000002 2024-09-13T09:19:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018356+0000", 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2782999999999998e-05 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018403+0000", 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6726e-05 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33759 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1822720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311738750999993, 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873635000000002, 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33759 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993683+0000", 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993682+0000", 2024-09-13T09:19:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993685+0000", 2024-09-13T09:19:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:19:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993691+0000", 2024-09-13T09:19:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1840000000000004e-06 2024-09-13T09:19:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993696+0000", 2024-09-13T09:19:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:19:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018429+0000", 2024-09-13T09:19:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024733330000000001 2024-09-13T09:19:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018494+0000", 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5233000000000004e-05 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018556+0000", 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2125999999999994e-05 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33760 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311721895999995, 2024-09-13T09:19:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025058980000000002, 2024-09-13T09:19:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33760 2024-09-13T09:19:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993700+0000", 2024-09-13T09:19:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993699+0000", 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993702+0000", 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.632e-06 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993708+0000", 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2280000000000003e-06 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993713+0000", 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018586+0000", 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024873572 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018645+0000", 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9098000000000002e-05 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018759+0000", 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113224 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33761 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1880064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311705199000002, 2024-09-13T09:19:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025229076, 2024-09-13T09:19:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33761 2024-09-13T09:19:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993716+0000", 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993718+0000", 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.092e-06 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993724+0000", 2024-09-13T09:19:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0839999999999997e-06 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993729+0000", 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018790+0000", 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025061039 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018871+0000", 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0885999999999998e-05 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.018945+0000", 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4773999999999999e-05 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33762 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311688724999996, 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025559609, 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33762 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993733+0000", 2024-09-13T09:19:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993732+0000", 2024-09-13T09:19:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993735+0000", 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993740+0000", 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993745+0000", 2024-09-13T09:19:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5290000000000002e-06 2024-09-13T09:19:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019043+0000", 2024-09-13T09:19:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025298355000000002 2024-09-13T09:19:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019134+0000", 2024-09-13T09:19:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0781999999999999e-05 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019292+0000", 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015829 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33763 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1937408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311672783999995, 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025732657999999999, 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33763 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993749+0000", 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993748+0000", 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993755+0000", 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993762+0000", 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993767+0000", 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7230000000000004e-06 2024-09-13T09:19:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019322+0000", 2024-09-13T09:19:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025555564999999999 2024-09-13T09:19:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019402+0000", 2024-09-13T09:19:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9433999999999999e-05 2024-09-13T09:19:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019481+0000", 2024-09-13T09:19:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9739000000000003e-05 2024-09-13T09:19:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33764 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311650409999999, 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025908526000000001, 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33764 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993771+0000", 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993773+0000", 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993779+0000", 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9610000000000002e-06 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993783+0000", 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.459e-06 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019513+0000", 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025729399 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019609+0000", 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6504999999999998e-05 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019680+0000", 2024-09-13T09:19:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0393000000000002e-05 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33765 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 1994752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311634174999995, 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026051009999999999, 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33765 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993787+0000", 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993789+0000", 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993795+0000", 2024-09-13T09:19:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.088e-06 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993799+0000", 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019704+0000", 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025904311999999999 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019766+0000", 2024-09-13T09:19:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2829000000000004e-05 2024-09-13T09:19:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019838+0000", 2024-09-13T09:19:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1935999999999997e-05 2024-09-13T09:19:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33766 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311618120000006, 2024-09-13T09:19:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026238971999999999, 2024-09-13T09:19:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33766 2024-09-13T09:19:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993803+0000", 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993805+0000", 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993811+0000", 2024-09-13T09:19:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9000000000000003e-06 2024-09-13T09:19:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993816+0000", 2024-09-13T09:19:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.527e-06 2024-09-13T09:19:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019877+0000", 2024-09-13T09:19:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026061337 2024-09-13T09:19:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.019960+0000", 2024-09-13T09:19:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3245999999999995e-05 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020042+0000", 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2053999999999997e-05 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33767 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2052096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311601710000005, 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026407013999999999, 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33767 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993820+0000", 2024-09-13T09:19:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993819+0000", 2024-09-13T09:19:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993822+0000", 2024-09-13T09:19:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:19:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993828+0000", 2024-09-13T09:19:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:19:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993832+0000", 2024-09-13T09:19:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.493e-06 2024-09-13T09:19:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020072+0000", 2024-09-13T09:19:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026239519999999999 2024-09-13T09:19:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020151+0000", 2024-09-13T09:19:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9278999999999998e-05 2024-09-13T09:19:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020227+0000", 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5965999999999997e-05 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:33768 3.6 3:61d81146:::rbd_data.1185833655de.0000000000000010:16 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.311583256999995, 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026564424, 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 33768 2024-09-13T09:19:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993838+0000", 2024-09-13T09:19:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993840+0000", 2024-09-13T09:19:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.996e-06 2024-09-13T09:19:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993846+0000", 2024-09-13T09:19:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:19:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.993858+0000", 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2214e-05 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020253+0000", 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026394926999999999 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020329+0000", 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5193999999999999e-05 2024-09-13T09:19:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.020403+0000", 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4145000000000002e-05 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:78829 3.6 3:6418b130:::rbd_data.11853dd604c.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 50.088897062999997, 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.041141776999999997, 2024-09-13T09:19:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 78829 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216524+0000", 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216522+0000", 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216540+0000", 2024-09-13T09:19:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8280000000000001e-05 2024-09-13T09:19:49.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216577+0000", 2024-09-13T09:19:49.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7151999999999999e-05 2024-09-13T09:19:49.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216594+0000", 2024-09-13T09:19:49.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7113e-05 2024-09-13T09:19:49.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216696+0000", 2024-09-13T09:19:49.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010190400000000001 2024-09-13T09:19:49.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.216833+0000", 2024-09-13T09:19:49.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136697 2024-09-13T09:19:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.224657+0000", 2024-09-13T09:19:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078241000000000005 2024-09-13T09:19:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257365+0000", 2024-09-13T09:19:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032707984000000002 2024-09-13T09:19:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257404+0000", 2024-09-13T09:19:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8979000000000001e-05 2024-09-13T09:19:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257549+0000", 2024-09-13T09:19:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000145737 2024-09-13T09:19:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:59.257666+0000", 2024-09-13T09:19:49.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116754 2024-09-13T09:19:49.409 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.409 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.409 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.409 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.409 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.409 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:49.410 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:49.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:19:49.412 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:49.412 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:49.413 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:49.414 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:49.414 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:49.403+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:49.512 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:49.507+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:49.581 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:49.581 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:49.581 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-13T09:19:49.581 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:49.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:19:49.613 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:49.609+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:49.714 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:49.707+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:49.778 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-13T09:19:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-13T09:19:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-13T09:19:49.779 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:49.796 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:19:49.814 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:49.813+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:49.916 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-13T09:19:49.913+0000 7fefaab97640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26215) UID: 0 2024-09-13T09:19:49.971 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:19:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-13T09:19:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-13T09:19:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-13T09:19:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:14584 3.2 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13893632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:19:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 122.43302727, 2024-09-13T09:19:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.060636556000000001, 2024-09-13T09:19:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 14584 2024-09-13T09:19:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536361+0000", 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.536357+0000", 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542055+0000", 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005697321 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542101+0000", 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5924000000000003e-05 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542119+0000", 2024-09-13T09:19:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8817999999999998e-05 2024-09-13T09:19:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542215+0000", 2024-09-13T09:19:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5987000000000001e-05 2024-09-13T09:19:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.542465+0000", 2024-09-13T09:19:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024903799999999998 2024-09-13T09:19:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-13T09:19:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.544021+0000", 2024-09-13T09:19:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001556209 2024-09-13T09:19:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.558391+0000", 2024-09-13T09:19:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014370374999999999 2024-09-13T09:19:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596897+0000", 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038505983000000001 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596944+0000", 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6928999999999997e-05 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:17:47.596997+0000", 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3060999999999999e-05 2024-09-13T09:19:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32900 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063460285000005, 2024-09-13T09:19:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046152506000000003, 2024-09-13T09:19:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32900 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905928+0000", 2024-09-13T09:19:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905927+0000", 2024-09-13T09:19:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905930+0000", 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905936+0000", 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2160000000000003e-06 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905941+0000", 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7480000000000001e-06 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952070+0000", 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046129772999999999 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952080+0000", 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6730000000000004e-06 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32901 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063442964000004, 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046154339000000003, 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32901 2024-09-13T09:19:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905945+0000", 2024-09-13T09:19:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905944+0000", 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905947+0000", 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905953+0000", 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1140000000000002e-06 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905958+0000", 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952095+0000", 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046136822000000001 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952099+0000", 2024-09-13T09:19:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:19:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32902 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063426023000005, 2024-09-13T09:19:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046167079, 2024-09-13T09:19:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32902 2024-09-13T09:19:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905962+0000", 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905961+0000", 2024-09-13T09:19:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905964+0000", 2024-09-13T09:19:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.227e-06 2024-09-13T09:19:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905970+0000", 2024-09-13T09:19:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2190000000000001e-06 2024-09-13T09:19:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905974+0000", 2024-09-13T09:19:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:19:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952119+0000", 2024-09-13T09:19:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046144778999999997 2024-09-13T09:19:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952129+0000", 2024-09-13T09:19:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6299999999999993e-06 2024-09-13T09:19:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32903 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063408795000001, 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046204736000000003, 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32903 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905979+0000", 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905981+0000", 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905987+0000", 2024-09-13T09:19:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:19:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.905992+0000", 2024-09-13T09:19:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:19:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952173+0000", 2024-09-13T09:19:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046180624000000003 2024-09-13T09:19:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952184+0000", 2024-09-13T09:19:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1117e-05 2024-09-13T09:19:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32904 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063372373999997, 2024-09-13T09:19:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046220926000000002, 2024-09-13T09:19:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32904 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906015+0000", 2024-09-13T09:19:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906036+0000", 2024-09-13T09:19:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0596000000000001e-05 2024-09-13T09:19:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906045+0000", 2024-09-13T09:19:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5240000000000006e-06 2024-09-13T09:19:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906051+0000", 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0249999999999999e-06 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952225+0000", 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046174245000000003 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952236+0000", 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0991e-05 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32905 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063318912, 2024-09-13T09:19:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046202957000000003, 2024-09-13T09:19:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32905 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906069+0000", 2024-09-13T09:19:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906068+0000", 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906072+0000", 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906079+0000", 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.52e-06 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906085+0000", 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952266+0000", 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046181497000000002 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952272+0000", 2024-09-13T09:19:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.378e-06 2024-09-13T09:19:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32906 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063296891999997, 2024-09-13T09:19:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214984000000001, 2024-09-13T09:19:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32906 2024-09-13T09:19:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906091+0000", 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906090+0000", 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906093+0000", 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8720000000000002e-06 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906105+0000", 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1576999999999999e-05 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906111+0000", 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8479999999999997e-06 2024-09-13T09:19:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952299+0000", 2024-09-13T09:19:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046188090000000001 2024-09-13T09:19:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952306+0000", 2024-09-13T09:19:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0899999999999999e-06 2024-09-13T09:19:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32907 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063272256999994, 2024-09-13T09:19:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046208120999999998, 2024-09-13T09:19:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32907 2024-09-13T09:19:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906116+0000", 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906115+0000", 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906118+0000", 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906124+0000", 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906130+0000", 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952319+0000", 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046189626999999997 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952324+0000", 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3449999999999999e-06 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32908 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063253734, 2024-09-13T09:19:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046217134999999999, 2024-09-13T09:19:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32908 2024-09-13T09:19:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906134+0000", 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906136+0000", 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906143+0000", 2024-09-13T09:19:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.297e-06 2024-09-13T09:19:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906148+0000", 2024-09-13T09:19:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:19:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952341+0000", 2024-09-13T09:19:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046193397999999997 2024-09-13T09:19:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952351+0000", 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9040000000000004e-06 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32909 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063235863000003, 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046216791, 2024-09-13T09:19:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32909 2024-09-13T09:19:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906152+0000", 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906154+0000", 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.593e-06 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906160+0000", 2024-09-13T09:19:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906165+0000", 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5900000000000001e-06 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952363+0000", 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046198515000000002 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952369+0000", 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32910 3.18 3.db17f7f8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063218532999997, 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046214429000000001, 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32910 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906169+0000", 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906171+0000", 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906177+0000", 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906183+0000", 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952379+0000", 2024-09-13T09:19:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046196096999999998 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952384+0000", 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32943 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063201062999994, 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046219291000000003, 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32943 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906187+0000", 2024-09-13T09:19:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906186+0000", 2024-09-13T09:19:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906189+0000", 2024-09-13T09:19:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2570000000000002e-06 2024-09-13T09:19:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906195+0000", 2024-09-13T09:19:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:19:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906200+0000", 2024-09-13T09:19:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:19:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952401+0000", 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046201751999999999 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952406+0000", 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32944 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063184196999998, 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046242951999999997, 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32944 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906204+0000", 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906203+0000", 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906206+0000", 2024-09-13T09:19:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.295e-06 2024-09-13T09:19:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906212+0000", 2024-09-13T09:19:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:19:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906217+0000", 2024-09-13T09:19:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0780000000000003e-06 2024-09-13T09:19:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952436+0000", 2024-09-13T09:19:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046218980999999999 2024-09-13T09:19:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952447+0000", 2024-09-13T09:19:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0556e-05 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32945 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063166409999994, 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046240303000000003, 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32945 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906221+0000", 2024-09-13T09:19:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906223+0000", 2024-09-13T09:19:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:19:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906253+0000", 2024-09-13T09:19:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9972999999999999e-05 2024-09-13T09:19:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906260+0000", 2024-09-13T09:19:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:19:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952457+0000", 2024-09-13T09:19:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046197290000000002 2024-09-13T09:19:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952462+0000", 2024-09-13T09:19:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4379999999999997e-06 2024-09-13T09:19:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:32946 3.18 3.f06bba38 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.063122100000001, 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046235631999999999, 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 32946 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906266+0000", 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906265+0000", 2024-09-13T09:19:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906269+0000", 2024-09-13T09:19:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4809999999999998e-06 2024-09-13T09:19:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906278+0000", 2024-09-13T09:19:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7850000000000003e-06 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.906284+0000", 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.891e-06 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952492+0000", 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046207491000000003 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:16.952501+0000", 2024-09-13T09:19:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-13T09:19:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35711 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.829991612000001, 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046356322999999998, 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35711 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139396+0000", 2024-09-13T09:19:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139398+0000", 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139404+0000", 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139409+0000", 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185534+0000", 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046125446000000001 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185637+0000", 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102957 2024-09-13T09:19:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185753+0000", 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011506 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35712 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 229376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.829974778999997, 2024-09-13T09:19:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046522991, 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35712 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139413+0000", 2024-09-13T09:19:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139415+0000", 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.204e-06 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139420+0000", 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139425+0000", 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185794+0000", 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046368841000000001 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185848+0000", 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.185936+0000", 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8165999999999996e-05 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:35713 3.1e 3:7b5cff9c:::rbd_data.1185833655de.0000000000000030:16 [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 92.829958421000001, 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046771572999999997, 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-13T09:19:50.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35713 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139429+0000", 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139431+0000", 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139437+0000", 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.993e-06 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.139444+0000", 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.956e-06 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186016+0000", 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571525000000003 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186106+0000", 2024-09-13T09:19:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0313000000000001e-05 2024-09-13T09:19:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:17.186201+0000", 2024-09-13T09:19:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4858000000000006e-05 2024-09-13T09:19:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4485.0:66592 3.7 3:e0817996:::rbd_data.1185b1e141e2.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:19:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.325598690999996, 2024-09-13T09:19:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.051168777999999998, 2024-09-13T09:19:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-13T09:19:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-13T09:19:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4485", 2024-09-13T09:19:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 66592 2024-09-13T09:19:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-13T09:19:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-13T09:19:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-13T09:19:50.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643789+0000", 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643787+0000", 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643799+0000", 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2140000000000001e-05 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643827+0000", 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8121999999999999e-05 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643841+0000", 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3969e-05 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.643985+0000", 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014404199999999999 2024-09-13T09:19:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.644371+0000", 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038588700000000001 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.654317+0000", 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099452050000000004 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694641+0000", 2024-09-13T09:19:50.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.040324802999999999 2024-09-13T09:19:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-13T09:19:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694682+0000", 2024-09-13T09:19:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0451999999999998e-05 2024-09-13T09:19:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-13T09:19:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694828+0000", 2024-09-13T09:19:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014580499999999999 2024-09-13T09:19:50.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-13T09:19:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-13T09:19:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-13T09:19:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-13T09:18:39.694958+0000", 2024-09-13T09:19:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013029599999999999 2024-09-13T09:19:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-13T09:19:50.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-13T09:19:50.071 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:19:50.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:19:50.073 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:50.069+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:50.174 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:50.173+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:50.200 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:50.200 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:50.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:19:50.306 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:50.305+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:50.376 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:50.371+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:50.389 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:50.389 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:50.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:19:50.477 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:50.471+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:50.574 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:50.574 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:50.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:50.574 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:50.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.574 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4391 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.574 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.586342239, 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069671680999999999, 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4391 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985967+0000", 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985966+0000", 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985972+0000", 2024-09-13T09:19:50.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:19:50.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.985990+0000", 2024-09-13T09:19:50.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7954000000000001e-05 2024-09-13T09:19:50.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986004+0000", 2024-09-13T09:19:50.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845999999999999e-05 2024-09-13T09:19:50.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055512+0000", 2024-09-13T09:19:50.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069508189999999997 2024-09-13T09:19:50.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055572+0000", 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0306000000000002e-05 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055639+0000", 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6588999999999996e-05 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4392 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.579 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:50.580 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.58629350299999, 2024-09-13T09:19:50.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069805685000000006, 2024-09-13T09:19:50.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.580 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4392 2024-09-13T09:19:50.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.580 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986016+0000", 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986015+0000", 2024-09-13T09:19:50.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986036+0000", 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1353e-05 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986056+0000", 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9239999999999999e-05 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986071+0000", 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4904e-05 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055670+0000", 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069599312999999996 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055737+0000", 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7534999999999996e-05 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055822+0000", 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4393 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.586225834, 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069908676000000003, 2024-09-13T09:19:50.584 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4393 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986084+0000", 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986083+0000", 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986089+0000", 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986107+0000", 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986121+0000", 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4185999999999999e-05 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055856+0000", 2024-09-13T09:19:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069734255999999994 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055916+0000", 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0905e-05 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.055992+0000", 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5854999999999998e-05 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4394 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.586085625, 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069977473999999998, 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4394 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.589 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986224+0000", 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986222+0000", 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986231+0000", 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986260+0000", 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8203000000000001e-05 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986300+0000", 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0289e-05 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056046+0000", 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069745636 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056115+0000", 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8931999999999997e-05 2024-09-13T09:19:50.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056201+0000", 2024-09-13T09:19:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6749999999999994e-05 2024-09-13T09:19:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4395 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.585994759, 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070079249999999996, 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4395 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986315+0000", 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986314+0000", 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986321+0000", 2024-09-13T09:19:50.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3429999999999998e-06 2024-09-13T09:19:50.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986341+0000", 2024-09-13T09:19:50.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9786999999999999e-05 2024-09-13T09:19:50.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986354+0000", 2024-09-13T09:19:50.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3699999999999999e-05 2024-09-13T09:19:50.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056248+0000", 2024-09-13T09:19:50.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069893828000000005 2024-09-13T09:19:50.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056314+0000", 2024-09-13T09:19:50.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6332000000000001e-05 2024-09-13T09:19:50.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056394+0000", 2024-09-13T09:19:50.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9430000000000004e-05 2024-09-13T09:19:50.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.600 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.600 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4396 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.600 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:50.600 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.585919899, 2024-09-13T09:19:50.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070278577999999994, 2024-09-13T09:19:50.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4396 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986390+0000", 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986388+0000", 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986395+0000", 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.703e-06 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986415+0000", 2024-09-13T09:19:50.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9587000000000001e-05 2024-09-13T09:19:50.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986448+0000", 2024-09-13T09:19:50.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2826e-05 2024-09-13T09:19:50.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056435+0000", 2024-09-13T09:19:50.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069987509000000003 2024-09-13T09:19:50.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056516+0000", 2024-09-13T09:19:50.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0803999999999994e-05 2024-09-13T09:19:50.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056668+0000", 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152206 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4397 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.585848723, 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070440997000000005, 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4397 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986461+0000", 2024-09-13T09:19:50.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986460+0000", 2024-09-13T09:19:50.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986466+0000", 2024-09-13T09:19:50.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:19:50.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986485+0000", 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8533999999999999e-05 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986498+0000", 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3056e-05 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056720+0000", 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070221993999999996 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056806+0000", 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6099000000000004e-05 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056902+0000", 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5767999999999995e-05 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.609 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.610 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4398 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.610 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:50.610 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.58579967, 2024-09-13T09:19:50.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070631343999999999, 2024-09-13T09:19:50.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4398 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986510+0000", 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986509+0000", 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986514+0000", 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4e-06 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986532+0000", 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7708999999999999e-05 2024-09-13T09:19:50.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986546+0000", 2024-09-13T09:19:50.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3407000000000001e-05 2024-09-13T09:19:50.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.056950+0000", 2024-09-13T09:19:50.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070404949999999994 2024-09-13T09:19:50.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057029+0000", 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9027000000000001e-05 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057141+0000", 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011162099999999999 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.615 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4399 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.615 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:50.615 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.58573793399999, 2024-09-13T09:19:50.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070847073999999996, 2024-09-13T09:19:50.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4399 2024-09-13T09:19:50.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986572+0000", 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986571+0000", 2024-09-13T09:19:50.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986577+0000", 2024-09-13T09:19:50.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:19:50.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986596+0000", 2024-09-13T09:19:50.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8817e-05 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986609+0000", 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.328e-05 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057190+0000", 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070580553000000004 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057273+0000", 2024-09-13T09:19:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3119000000000002e-05 2024-09-13T09:19:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057419+0000", 2024-09-13T09:19:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014588200000000001 2024-09-13T09:19:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4400 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.585688264, 2024-09-13T09:19:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071006680000000003, 2024-09-13T09:19:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4400 2024-09-13T09:19:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:50.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986621+0000", 2024-09-13T09:19:50.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986620+0000", 2024-09-13T09:19:50.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986626+0000", 2024-09-13T09:19:50.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:19:50.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986644+0000", 2024-09-13T09:19:50.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7983e-05 2024-09-13T09:19:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986658+0000", 2024-09-13T09:19:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:19:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057466+0000", 2024-09-13T09:19:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070808259999999998 2024-09-13T09:19:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057542+0000", 2024-09-13T09:19:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5556999999999995e-05 2024-09-13T09:19:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057628+0000", 2024-09-13T09:19:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6126999999999997e-05 2024-09-13T09:19:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.625 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4401 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.625 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:50.625 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.58563974499999, 2024-09-13T09:19:50.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071174813000000003, 2024-09-13T09:19:50.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.625 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.625 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.625 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.625 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.625 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4401 2024-09-13T09:19:50.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.626 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:50.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986670+0000", 2024-09-13T09:19:50.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986669+0000", 2024-09-13T09:19:50.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986674+0000", 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986692+0000", 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986706+0000", 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3837e-05 2024-09-13T09:19:50.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057661+0000", 2024-09-13T09:19:50.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070955275999999998 2024-09-13T09:19:50.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057738+0000", 2024-09-13T09:19:50.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7185e-05 2024-09-13T09:19:50.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057845+0000", 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010600899999999999 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4402 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.585591349, 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07132028, 2024-09-13T09:19:50.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4402 2024-09-13T09:19:50.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.631 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:50.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986718+0000", 2024-09-13T09:19:50.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986717+0000", 2024-09-13T09:19:50.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986723+0000", 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.507e-06 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986740+0000", 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7796999999999999e-05 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986753+0000", 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3053e-05 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057896+0000", 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071142993000000002 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.057960+0000", 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3788000000000007e-05 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058038+0000", 2024-09-13T09:19:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8152000000000007e-05 2024-09-13T09:19:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4403 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.585544062, 2024-09-13T09:19:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071504705000000002, 2024-09-13T09:19:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4403 2024-09-13T09:19:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986765+0000", 2024-09-13T09:19:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986764+0000", 2024-09-13T09:19:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986770+0000", 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986820+0000", 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9749000000000001e-05 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986835+0000", 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.518e-05 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058084+0000", 2024-09-13T09:19:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071248594999999998 2024-09-13T09:19:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058178+0000", 2024-09-13T09:19:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4041000000000003e-05 2024-09-13T09:19:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058270+0000", 2024-09-13T09:19:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2390000000000001e-05 2024-09-13T09:19:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4404 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.58546066, 2024-09-13T09:19:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071622398000000004, 2024-09-13T09:19:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4404 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986849+0000", 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986848+0000", 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986872+0000", 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.404e-05 2024-09-13T09:19:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986892+0000", 2024-09-13T09:19:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0034000000000002e-05 2024-09-13T09:19:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986906+0000", 2024-09-13T09:19:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4178999999999999e-05 2024-09-13T09:19:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058316+0000", 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071410290000000001 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058385+0000", 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8367000000000005e-05 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058471+0000", 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6648000000000001e-05 2024-09-13T09:19:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4405 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.585386934, 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071785179000000005, 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4405 2024-09-13T09:19:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986923+0000", 2024-09-13T09:19:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986921+0000", 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986928+0000", 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986945+0000", 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7289999999999999e-05 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986977+0000", 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2406000000000001e-05 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058527+0000", 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071549132000000001 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058616+0000", 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9705999999999998e-05 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058708+0000", 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1417999999999998e-05 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4406 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.58531881099999, 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071922978999999998, 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4406 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986991+0000", 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986990+0000", 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.986996+0000", 2024-09-13T09:19:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4999999999999996e-06 2024-09-13T09:19:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987015+0000", 2024-09-13T09:19:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8899999999999999e-05 2024-09-13T09:19:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987029+0000", 2024-09-13T09:19:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3773e-05 2024-09-13T09:19:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058754+0000", 2024-09-13T09:19:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071725211999999997 2024-09-13T09:19:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058826+0000", 2024-09-13T09:19:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2298000000000005e-05 2024-09-13T09:19:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058914+0000", 2024-09-13T09:19:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7329000000000004e-05 2024-09-13T09:19:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:4409 3.3 3:c5f53f78:::rbd_data.1185192b47c4.0000000000000005:5 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:19:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.585268344, 2024-09-13T09:19:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072094781999999996, 2024-09-13T09:19:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 4409 2024-09-13T09:19:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987041+0000", 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987040+0000", 2024-09-13T09:19:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987046+0000", 2024-09-13T09:19:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5400000000000003e-06 2024-09-13T09:19:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987069+0000", 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3708999999999999e-05 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:43.987083+0000", 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3764000000000001e-05 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.058965+0000", 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.071881686 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059050+0000", 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5100999999999997e-05 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:44.059136+0000", 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6028999999999999e-05 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:78833 3.a 3:5235bc0b:::rbd_data.11853dd604c.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.302998527, 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.35280560599999999, 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78833 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269311+0000", 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269309+0000", 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-13T09:19:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269322+0000", 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3359e-05 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269352+0000", 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0320000000000001e-05 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269367+0000", 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.526e-05 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269434+0000", 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.648e-05 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.269446+0000", 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2286999999999999e-05 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.606711+0000", 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.33726426399999998 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.606879+0000", 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016884100000000001 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.609371+0000", 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024920110000000001 2024-09-13T09:19:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622034+0000", 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012662199000000001 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622042+0000", 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9570000000000002e-06 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622083+0000", 2024-09-13T09:19:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.197e-05 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.622117+0000", 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3013999999999999e-05 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:80090 3.a 3:517b773f:::rbd_object_map.118553778f1e:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.846625230999997, 2024-09-13T09:19:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99537752499999999, 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80090 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725684+0000", 2024-09-13T09:19:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725681+0000", 2024-09-13T09:19:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-13T09:19:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725699+0000", 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7683999999999999e-05 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725746+0000", 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7196e-05 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725769+0000", 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3383e-05 2024-09-13T09:19:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725956+0000", 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000187049 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:07.725985+0000", 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8472000000000001e-05 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.718483+0000", 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99249815200000002 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.718851+0000", 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036844299999999998 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.719201+0000", 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034928699999999999 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720843+0000", 2024-09-13T09:19:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016425249999999999 2024-09-13T09:19:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720882+0000", 2024-09-13T09:19:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9045999999999999e-05 2024-09-13T09:19:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.720985+0000", 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103163 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:08.721062+0000", 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6402000000000005e-05 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:81122 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:19:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.671784817999999, 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.920409343, 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 81122 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900525+0000", 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900523+0000", 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900536+0000", 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3200000000000001e-05 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900565+0000", 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9703e-05 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900588+0000", 2024-09-13T09:19:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2096e-05 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900698+0000", 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110029 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:16.900716+0000", 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8070000000000001e-05 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:17.766722+0000", 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.86600628099999999 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:17.766770+0000", 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8486000000000003e-05 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.817359+0000", 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.050588836 2024-09-13T09:19:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.817852+0000", 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00049274599999999996 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.818512+0000", 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00066035099999999999 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820216+0000", 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017038680000000001 2024-09-13T09:19:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820601+0000", 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038506599999999999 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820746+0000", 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014458900000000001 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:18.820934+0000", 2024-09-13T09:19:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188102 2024-09-13T09:19:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:50.681 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:50.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:19:50.683 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:50.681+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:50.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:19:50.796 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:50.791+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:50.863 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:50.863 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:50.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:19:50.884 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:50.879+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:50.985 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-13T09:19:50.979+0000 7f1a347b4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26169) UID: 0 2024-09-13T09:19:51.009 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:51.009 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:51.021 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 0, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:19:51.021 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:19:51.021 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-13T09:19:51.021 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:19:51.021 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:19:51.021 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-13T09:19:51.023 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:19:51.086 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-13T09:19:51.079+0000 7fd6fae13640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26171) UID: 0 2024-09-13T09:19:51.152 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:51.152 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "" 2024-09-13T09:19:51.152 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:51.187 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:51.183+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:51.189 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 142.50293792599999, 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.21372158699999999, 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685364+0000", 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685363+0000", 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685372+0000", 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4590000000000004e-06 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685394+0000", 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1616000000000001e-05 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685407+0000", 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3675e-05 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685484+0000", 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7065999999999997e-05 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.685493+0000", 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8070000000000006e-06 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690770+0000", 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052763869999999996 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.690940+0000", 2024-09-13T09:19:51.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017001900000000001 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.691238+0000", 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029794499999999998 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.692057+0000", 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081943000000000001 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.898933+0000", 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.20687588200000001 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899026+0000", 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2777999999999998e-05 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:28.899086+0000", 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0188e-05 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.196 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:14590 3.1d 3:be60f898:::rbd_data.11853ecd5f5f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13672152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 123.647089486, 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052557382, 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 14590 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541213+0000", 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.541207+0000", 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562477+0000", 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021270155999999998 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562539+0000", 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1851000000000001e-05 2024-09-13T09:19:51.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562565+0000", 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6194e-05 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.562702+0000", 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013731299999999999 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.563071+0000", 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036892899999999999 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.565167+0000", 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020961959999999998 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.585261+0000", 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020093443999999999 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593596+0000", 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083352849999999996 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593696+0000", 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100021 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.593770+0000", 2024-09-13T09:19:51.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3889999999999999e-05 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4485.0:14584 3.2 e39/22 3:4349d40d:::rbd_data.11853ecd5f5f.0000000000000001:head v 39'64, mlcod=38'62)", 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 123.64510384, 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051546091000000002, 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:19:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543198+0000", 2024-09-13T09:19:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.543197+0000", 2024-09-13T09:19:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588691+0000", 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045493475999999998 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588711+0000", 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0485000000000001e-05 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588721+0000", 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.499e-06 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588766+0000", 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5228000000000001e-05 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.588788+0000", 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2503000000000001e-05 2024-09-13T09:19:51.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594671+0000", 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058822809999999996 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:17:47.594744+0000", 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3776000000000007e-05 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:22580 3.1c 3:39876c9f:::rbd_data.1185d6f8ec87.000000000000000f:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 105.33585105, 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050372896, 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22580 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.208 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852451+0000", 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852449+0000", 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852462+0000", 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3756e-05 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852495+0000", 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2435999999999999e-05 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852516+0000", 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1347000000000001e-05 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852644+0000", 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012777200000000001 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.852784+0000", 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013980500000000001 2024-09-13T09:19:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.860936+0000", 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081526859999999993 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902522+0000", 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041585203000000001 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902560+0000", 2024-09-13T09:19:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8603000000000003e-05 2024-09-13T09:19:51.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:51.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902708+0000", 2024-09-13T09:19:51.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014740900000000001 2024-09-13T09:19:51.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:05.902824+0000", 2024-09-13T09:19:51.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116519 2024-09-13T09:19:51.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.213 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54987 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6447104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.945994792999997, 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041604369000000002, 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54987 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242307+0000", 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242309+0000", 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1160000000000002e-06 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242317+0000", 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9799999999999998e-06 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242323+0000", 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6439999999999997e-06 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283711+0000", 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388085999999998 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283766+0000", 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5112999999999999e-05 2024-09-13T09:19:51.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283912+0000", 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014575300000000001 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54988 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6545408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.945974973000006, 2024-09-13T09:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041865404000000002, 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54988 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242327+0000", 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242329+0000", 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242335+0000", 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242341+0000", 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9440000000000002e-06 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.283952+0000", 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041610981999999998 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284076+0000", 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000124356 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284193+0000", 2024-09-13T09:19:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116345 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54989 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6643712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.945956859000006, 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042123657000000002, 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54989 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242345+0000", 2024-09-13T09:19:51.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:51.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242348+0000", 2024-09-13T09:19:51.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.655e-06 2024-09-13T09:19:51.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242354+0000", 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2269999999999998e-06 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242360+0000", 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6170000000000003e-06 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284248+0000", 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041887830000000001 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284340+0000", 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1358000000000002e-05 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284469+0000", 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012932300000000001 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54990 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6742016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.945923569000001, 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042278333000000001, 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54990 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242379+0000", 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242378+0000", 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:51.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242381+0000", 2024-09-13T09:19:51.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0680000000000001e-06 2024-09-13T09:19:51.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242392+0000", 2024-09-13T09:19:51.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0762e-05 2024-09-13T09:19:51.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242398+0000", 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1040000000000003e-06 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284506+0000", 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042107983000000002 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284564+0000", 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.821e-05 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284657+0000", 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2588000000000002e-05 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54991 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6840320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.945898778, 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042489459, 2024-09-13T09:19:51.232 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.233 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.233 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.233 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.233 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.233 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54991 2024-09-13T09:19:51.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.236 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:19:51.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242403+0000", 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242406+0000", 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:19:51.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242412+0000", 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5919999999999997e-06 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242425+0000", 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2713e-05 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284694+0000", 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042269558999999998 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284784+0000", 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9999000000000003e-05 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284893+0000", 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108422 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54992 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 6938624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.945872406999996, 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042636897999999999, 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54992 2024-09-13T09:19:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242430+0000", 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242429+0000", 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242432+0000", 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7599999999999998e-06 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242439+0000", 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5760000000000002e-06 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242448+0000", 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284935+0000", 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042486949000000003 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.284991+0000", 2024-09-13T09:19:51.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6471000000000002e-05 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285067+0000", 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5543000000000005e-05 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54993 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7036928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.945849530000004, 2024-09-13T09:19:51.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042867959999999997, 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54993 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242453+0000", 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242452+0000", 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242455+0000", 2024-09-13T09:19:51.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4360000000000001e-06 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242463+0000", 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8220000000000007e-06 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242468+0000", 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9479999999999999e-06 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285107+0000", 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.042639415999999999 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285196+0000", 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8721000000000007e-05 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285321+0000", 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012495 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54994 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7135232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.945822366000002, 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043009622999999997, 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54994 2024-09-13T09:19:51.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242480+0000", 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242482+0000", 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242486+0000", 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1049999999999997e-06 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242489+0000", 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.004e-06 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285354+0000", 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.04286479 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285413+0000", 2024-09-13T09:19:51.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9778000000000003e-05 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285489+0000", 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6141000000000002e-05 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54995 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7233536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:19:51.253 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.945810995000002, 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043166641999999998, 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54995 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242491+0000", 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242492+0000", 2024-09-13T09:19:51.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.437e-06 2024-09-13T09:19:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242496+0000", 2024-09-13T09:19:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:19:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242498+0000", 2024-09-13T09:19:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5950000000000001e-06 2024-09-13T09:19:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285528+0000", 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043029755000000003 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285585+0000", 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285658+0000", 2024-09-13T09:19:51.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2713999999999995e-05 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54996 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7331840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.945801680000002, 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043333782000000001, 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.258 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54996 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242501+0000", 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242500+0000", 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242502+0000", 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3599999999999999e-06 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242505+0000", 2024-09-13T09:19:51.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1439999999999999e-06 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242507+0000", 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.605e-06 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285698+0000", 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043190911999999998 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285754+0000", 2024-09-13T09:19:51.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5541999999999998e-05 2024-09-13T09:19:51.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285834+0000", 2024-09-13T09:19:51.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0426000000000006e-05 2024-09-13T09:19:51.262 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.262 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.262 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54997 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.945792591, 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043496174999999998, 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54997 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242510+0000", 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242509+0000", 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-13T09:19:51.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242511+0000", 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.313e-06 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242514+0000", 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242516+0000", 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3999999999999999e-06 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285875+0000", 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043358578000000002 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.285929+0000", 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4602999999999999e-05 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286006+0000", 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6429999999999998e-05 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:54998 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 7528448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.945699978999997, 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043666469999999999, 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 54998 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242602+0000", 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242518+0000", 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999161 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242606+0000", 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7607000000000003e-05 2024-09-13T09:19:51.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242617+0000", 2024-09-13T09:19:51.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0882e-05 2024-09-13T09:19:51.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.242624+0000", 2024-09-13T09:19:51.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9009999999999997e-06 2024-09-13T09:19:51.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286046+0000", 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043422308 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286106+0000", 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0285000000000003e-05 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.286269+0000", 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000162578 2024-09-13T09:19:51.271 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:55006 3.16 3:6e2046df:::rbd_data.118568665621.0000000000000004:1d [read 8314880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.942311310999997, 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041461771000000001, 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 55006 2024-09-13T09:19:51.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.273 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:19:51.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:19:51.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245991+0000", 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245993+0000", 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2869999999999999e-06 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.245999+0000", 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0159999999999997e-06 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.246004+0000", 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287345+0000", 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041341217 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287398+0000", 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2840000000000002e-05 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:35.287453+0000", 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4746000000000001e-05 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:68662 3.1d 3:be8be2fa:::rbd_data.118556027ff4.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.031255869999995, 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.044923689000000003, 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 68662 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157046+0000", 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157045+0000", 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-13T09:19:51.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157052+0000", 2024-09-13T09:19:51.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8750000000000002e-06 2024-09-13T09:19:51.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157066+0000", 2024-09-13T09:19:51.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3815000000000001e-05 2024-09-13T09:19:51.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157073+0000", 2024-09-13T09:19:51.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.64e-06 2024-09-13T09:19:51.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157145+0000", 2024-09-13T09:19:51.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2517999999999999e-05 2024-09-13T09:19:51.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.157267+0000", 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012185599999999999 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.164012+0000", 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0067447660000000001 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201805+0000", 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037793528999999999 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201881+0000", 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5678000000000003e-05 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201933+0000", 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2346999999999999e-05 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:41.201970+0000", 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6532000000000002e-05 2024-09-13T09:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.283 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.283 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.283 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:78869 3.5 3:a7c01924:::rbd_data.11853dd604c.000000000000000b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-13T09:19:51.283 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:18:59.731157+0000", 2024-09-13T09:19:51.283 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.457145158000003, 2024-09-13T09:19:51.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062116486999999998, 2024-09-13T09:19:51.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:51.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78869 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731157+0000", 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731157+0000", 2024-09-13T09:19:51.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731155+0000", 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731171+0000", 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5543e-05 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731202+0000", 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1229999999999997e-05 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731217+0000", 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5173e-05 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731336+0000", 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119026 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.731566+0000", 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022948399999999999 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.742507+0000", 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010941383000000001 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.793038+0000", 2024-09-13T09:19:51.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050530670999999999 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.793060+0000", 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2002999999999999e-05 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.793155+0000", 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4716000000000006e-05 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:18:59.793274+0000", 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011882799999999999 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4485.0:79042 3.1c 3:3f2730ec:::rbd_data.11853dd604c.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-13T09:19:00.228566+0000", 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.959736024000001, 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066070161000000002, 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-13T09:19:51.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4485", 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/27216991", 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 79042 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228566+0000", 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228566+0000", 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228564+0000", 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228575+0000", 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-13T09:19:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228605+0000", 2024-09-13T09:19:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0426e-05 2024-09-13T09:19:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-13T09:19:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228620+0000", 2024-09-13T09:19:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4919999999999999e-05 2024-09-13T09:19:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-13T09:19:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228720+0000", 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9889000000000005e-05 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.228845+0000", 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012490600000000001 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.241066+0000", 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012220795 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.294366+0000", 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.053300301000000001 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.294481+0000", 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114629 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.294571+0000", 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0093000000000006e-05 2024-09-13T09:19:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-13T09:19:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-13T09:19:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-13T09:19:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-13T09:19:00.294636+0000", 2024-09-13T09:19:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5605999999999995e-05 2024-09-13T09:19:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-13T09:19:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-13T09:19:51.295 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:51.297 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:19:51.297 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:51.291+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:51.398 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:51.397+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:51.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:19:51.468 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:51.468 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:51.469 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-13T09:19:51.469 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:51.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:19:51.499 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-13T09:19:51.491+0000 7f3758f25640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26166) UID: 0 2024-09-13T09:19:51.575 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-13T09:19:51.575 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:19:51.593 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-13T09:19:51.593 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:19:51.599 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-13T09:19:51.597+0000 7ff9e370e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26219) UID: 0 2024-09-13T09:19:51.610 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-13T09:19:51.610 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:19:51.627 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-13T09:19:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:19:51.647 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-13T09:19:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:19:51.656 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-13T09:19:51.656 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-13T09:19:51.657 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-13T09:19:51.657 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-13T09:19:51.657 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-13T09:19:51.667 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-13T09:19:51.667 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:19:51.672 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:19:51.688 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-13T09:19:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:19:51.700 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-13T09:19:51.695+0000 7f97ac847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26173) UID: 0 2024-09-13T09:19:51.708 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-13T09:19:51.709 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:19:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-13T09:19:51.801 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-13T09:19:51.797+0000 7fdc6949e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26211) UID: 0 2024-09-13T09:19:51.835 INFO:teuthology.orchestra.run.smithi032.stdout:All operations completed A-OK! 2024-09-13T09:19:51.848 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:51.848 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:51.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:19:51.906 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-13T09:19:51.905+0000 7f2f79f91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26217) UID: 0 2024-09-13T09:19:51.917 DEBUG:teuthology.parallel:result is None 2024-09-13T09:19:51.917 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-13T09:19:51.929 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-13T09:19:51.947 INFO:tasks.thrashosds:joining thrashosds 2024-09-13T09:19:52.030 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:52.031 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:52.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:19:52.174 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:52.174 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:52.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:19:52.316 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-13T09:19:52.316 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:19:53.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:19:54.128 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-13T09:19:54.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-13T09:19:56.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-13T09:19:56.315 INFO:teuthology.orchestra.run.smithi032.stdout:mon.a: {} 2024-09-13T09:19:56.316 INFO:teuthology.orchestra.run.smithi032.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:19:56.328 INFO:teuthology.orchestra.run.smithi032.stdout:mon.b: {} 2024-09-13T09:19:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:19:56.338 INFO:teuthology.orchestra.run.smithi032.stdout:mon.c: {} 2024-09-13T09:19:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:19:56.356 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T09:19:56.714 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T09:19:56.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-13T09:19:57.717 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-13T09:19:57.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:19:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:19:58.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-13T09:19:59.731 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-13T09:19:59.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:20:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:20:00.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-13T09:20:01.753 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-13T09:20:01.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T09:20:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T09:20:02.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-13T09:20:03.772 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-13T09:20:03.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T09:20:04.783 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T09:20:04.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-13T09:20:05.793 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-13T09:20:05.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:20:06.803 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:20:06.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-13T09:20:07.815 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-13T09:20:07.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T09:20:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T09:20:08.857 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-13T09:20:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-13T09:20:09.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T09:20:10.846 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T09:20:10.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-13T09:20:11.863 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-13T09:20:11.887 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-13T09:20:11.887 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-13T09:20:11.887 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-13T09:20:11.887 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-13T09:20:11.887 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T09:20:11.887 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:20:12.192 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:20:12.192 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":167,"fsid":"f0889cb1-03d6-4b81-88c9-6141846c6033","created":"2024-09-13T09:16:56.964982+0000","modified":"2024-09-13T09:20:11.848833+0000","last_up_change":"2024-09-13T09:17:08.368378+0000","last_in_change":"2024-09-13T09:19:16.749126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:17:10.140920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:17:15.775544+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T09:17:25.804931+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":81,"snap_epoch":149,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"786bdfc4-cc25-4167-aa5b-3737cfeb4ac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6801","nonce":3549111487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6803","nonce":3549111487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6807","nonce":3549111487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6805","nonce":3549111487}]},"public_addr":"172.21.15.32:6801/3549111487","cluster_addr":"172.21.15.32:6803/3549111487","heartbeat_back_addr":"172.21.15.32:6807/3549111487","heartbeat_front_addr":"172.21.15.32:6805/3549111487","state":["exists","up"]},{"osd":1,"uuid":"ca6189ab-90dd-4e9d-a85f-092186153b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6810","nonce":1470371899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6815","nonce":1470371899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6823","nonce":1470371899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6820","nonce":1470371899}]},"public_addr":"172.21.15.32:6810/1470371899","cluster_addr":"172.21.15.32:6815/1470371899","heartbeat_back_addr":"172.21.15.32:6823/1470371899","heartbeat_front_addr":"172.21.15.32:6820/1470371899","state":["exists","up"]},{"osd":2,"uuid":"c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6825","nonce":2485123706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6827","nonce":2485123706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6831","nonce":2485123706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6829","nonce":2485123706}]},"public_addr":"172.21.15.32:6825/2485123706","cluster_addr":"172.21.15.32:6827/2485123706","heartbeat_back_addr":"172.21.15.32:6831/2485123706","heartbeat_front_addr":"172.21.15.32:6829/2485123706","state":["exists","up"]},{"osd":3,"uuid":"5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6811","nonce":1009890408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6814","nonce":1009890408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6821","nonce":1009890408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6818","nonce":1009890408}]},"public_addr":"172.21.15.32:6811/1009890408","cluster_addr":"172.21.15.32:6814/1009890408","heartbeat_back_addr":"172.21.15.32:6821/1009890408","heartbeat_front_addr":"172.21.15.32:6818/1009890408","state":["exists","up"]},{"osd":4,"uuid":"39f42e87-29dd-42cc-9ef6-e0074f1794e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6823","nonce":2723023689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6825","nonce":2723023689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6831","nonce":2723023689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6828","nonce":2723023689}]},"public_addr":"172.21.15.46:6823/2723023689","cluster_addr":"172.21.15.46:6825/2723023689","heartbeat_back_addr":"172.21.15.46:6831/2723023689","heartbeat_front_addr":"172.21.15.46:6828/2723023689","state":["exists","up"]},{"osd":5,"uuid":"cf8dd701-9543-4bdd-a6f6-0b56210f00c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6809","nonce":2955019640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6812","nonce":2955019640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2955019640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6815","nonce":2955019640}]},"public_addr":"172.21.15.46:6809/2955019640","cluster_addr":"172.21.15.46:6812/2955019640","heartbeat_back_addr":"172.21.15.46:6819/2955019640","heartbeat_front_addr":"172.21.15.46:6815/2955019640","state":["exists","up"]},{"osd":6,"uuid":"dd080b88-3292-4a34-8dbd-645683d8fe14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6813","nonce":916160151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6818","nonce":916160151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6829","nonce":916160151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6822","nonce":916160151}]},"public_addr":"172.21.15.46:6813/916160151","cluster_addr":"172.21.15.46:6818/916160151","heartbeat_back_addr":"172.21.15.46:6829/916160151","heartbeat_front_addr":"172.21.15.46:6822/916160151","state":["exists","up"]},{"osd":7,"uuid":"c6bca54b-e982-4d6a-9ae9-057ee3881a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6801","nonce":3477167689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6803","nonce":3477167689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6807","nonce":3477167689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6805","nonce":3477167689}]},"public_addr":"172.21.15.46:6801/3477167689","cluster_addr":"172.21.15.46:6803/3477167689","heartbeat_back_addr":"172.21.15.46:6807/3477167689","heartbeat_front_addr":"172.21.15.46:6805/3477167689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.247113+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.166872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.451769+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.458130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.466129+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.444828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.425145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.484371+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:20:12.212 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T09:20:12.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:20:12.519 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:20:12.520 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":167,"fsid":"f0889cb1-03d6-4b81-88c9-6141846c6033","created":"2024-09-13T09:16:56.964982+0000","modified":"2024-09-13T09:20:11.848833+0000","last_up_change":"2024-09-13T09:17:08.368378+0000","last_in_change":"2024-09-13T09:19:16.749126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:17:10.140920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:17:15.775544+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T09:17:25.804931+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":81,"snap_epoch":149,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"786bdfc4-cc25-4167-aa5b-3737cfeb4ac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6801","nonce":3549111487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6803","nonce":3549111487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6807","nonce":3549111487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6805","nonce":3549111487}]},"public_addr":"172.21.15.32:6801/3549111487","cluster_addr":"172.21.15.32:6803/3549111487","heartbeat_back_addr":"172.21.15.32:6807/3549111487","heartbeat_front_addr":"172.21.15.32:6805/3549111487","state":["exists","up"]},{"osd":1,"uuid":"ca6189ab-90dd-4e9d-a85f-092186153b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6810","nonce":1470371899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6815","nonce":1470371899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6823","nonce":1470371899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6820","nonce":1470371899}]},"public_addr":"172.21.15.32:6810/1470371899","cluster_addr":"172.21.15.32:6815/1470371899","heartbeat_back_addr":"172.21.15.32:6823/1470371899","heartbeat_front_addr":"172.21.15.32:6820/1470371899","state":["exists","up"]},{"osd":2,"uuid":"c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6825","nonce":2485123706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6827","nonce":2485123706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6831","nonce":2485123706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6829","nonce":2485123706}]},"public_addr":"172.21.15.32:6825/2485123706","cluster_addr":"172.21.15.32:6827/2485123706","heartbeat_back_addr":"172.21.15.32:6831/2485123706","heartbeat_front_addr":"172.21.15.32:6829/2485123706","state":["exists","up"]},{"osd":3,"uuid":"5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6811","nonce":1009890408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6814","nonce":1009890408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6821","nonce":1009890408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6818","nonce":1009890408}]},"public_addr":"172.21.15.32:6811/1009890408","cluster_addr":"172.21.15.32:6814/1009890408","heartbeat_back_addr":"172.21.15.32:6821/1009890408","heartbeat_front_addr":"172.21.15.32:6818/1009890408","state":["exists","up"]},{"osd":4,"uuid":"39f42e87-29dd-42cc-9ef6-e0074f1794e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6823","nonce":2723023689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6825","nonce":2723023689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6831","nonce":2723023689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6828","nonce":2723023689}]},"public_addr":"172.21.15.46:6823/2723023689","cluster_addr":"172.21.15.46:6825/2723023689","heartbeat_back_addr":"172.21.15.46:6831/2723023689","heartbeat_front_addr":"172.21.15.46:6828/2723023689","state":["exists","up"]},{"osd":5,"uuid":"cf8dd701-9543-4bdd-a6f6-0b56210f00c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6809","nonce":2955019640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6812","nonce":2955019640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2955019640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6815","nonce":2955019640}]},"public_addr":"172.21.15.46:6809/2955019640","cluster_addr":"172.21.15.46:6812/2955019640","heartbeat_back_addr":"172.21.15.46:6819/2955019640","heartbeat_front_addr":"172.21.15.46:6815/2955019640","state":["exists","up"]},{"osd":6,"uuid":"dd080b88-3292-4a34-8dbd-645683d8fe14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6813","nonce":916160151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6818","nonce":916160151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6829","nonce":916160151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6822","nonce":916160151}]},"public_addr":"172.21.15.46:6813/916160151","cluster_addr":"172.21.15.46:6818/916160151","heartbeat_back_addr":"172.21.15.46:6829/916160151","heartbeat_front_addr":"172.21.15.46:6822/916160151","state":["exists","up"]},{"osd":7,"uuid":"c6bca54b-e982-4d6a-9ae9-057ee3881a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6801","nonce":3477167689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6803","nonce":3477167689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6807","nonce":3477167689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6805","nonce":3477167689}]},"public_addr":"172.21.15.46:6801/3477167689","cluster_addr":"172.21.15.46:6803/3477167689","heartbeat_back_addr":"172.21.15.46:6807/3477167689","heartbeat_front_addr":"172.21.15.46:6805/3477167689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.247113+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.166872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.451769+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.458130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.466129+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.444828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.425145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.484371+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:20:12.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T09:20:12.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T09:20:12.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T09:20:12.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T09:20:12.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T09:20:12.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T09:20:12.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T09:20:12.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T09:20:12.796 INFO:teuthology.orchestra.run.smithi032.stdout:55834574888 2024-09-13T09:20:12.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:20:12.797 INFO:teuthology.orchestra.run.smithi032.stdout:55834574889 2024-09-13T09:20:12.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:20:12.814 INFO:teuthology.orchestra.run.smithi032.stdout:55834574889 2024-09-13T09:20:12.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:20:12.817 INFO:teuthology.orchestra.run.smithi032.stdout:55834574888 2024-09-13T09:20:12.817 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:20:12.819 INFO:teuthology.orchestra.run.smithi032.stdout:55834574889 2024-09-13T09:20:12.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:20:12.821 INFO:teuthology.orchestra.run.smithi032.stdout:55834574888 2024-09-13T09:20:12.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:20:12.824 INFO:teuthology.orchestra.run.smithi032.stdout:55834574887 2024-09-13T09:20:12.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:20:12.828 INFO:teuthology.orchestra.run.smithi032.stdout:55834574887 2024-09-13T09:20:12.828 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:20:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:55834574887 2024-09-13T09:20:13.332 INFO:teuthology.orchestra.run.smithi032.stdout:55834574888 2024-09-13T09:20:13.336 INFO:teuthology.orchestra.run.smithi032.stdout:55834574887 2024-09-13T09:20:13.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574887 for osd.6 2024-09-13T09:20:13.351 INFO:teuthology.orchestra.run.smithi032.stdout:55834574888 2024-09-13T09:20:13.355 INFO:teuthology.orchestra.run.smithi032.stdout:55834574887 2024-09-13T09:20:13.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574889 got 55834574888 for osd.4 2024-09-13T09:20:13.362 INFO:teuthology.orchestra.run.smithi032.stdout:55834574888 2024-09-13T09:20:13.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574889 got 55834574888 for osd.2 2024-09-13T09:20:13.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574887 for osd.7 2024-09-13T09:20:13.384 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574889 got 55834574888 for osd.0 2024-09-13T09:20:13.386 INFO:teuthology.orchestra.run.smithi032.stdout:55834574886 2024-09-13T09:20:13.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574887 for osd.3 2024-09-13T09:20:13.401 INFO:teuthology.orchestra.run.smithi032.stdout:55834574886 2024-09-13T09:20:13.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574886 for osd.1 2024-09-13T09:20:13.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574886 for osd.5 2024-09-13T09:20:14.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:20:14.362 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:20:14.376 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:20:14.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:20:14.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:20:14.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:20:14.405 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:20:14.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:20:14.889 INFO:teuthology.orchestra.run.smithi032.stdout:55834574888 2024-09-13T09:20:14.919 INFO:teuthology.orchestra.run.smithi032.stdout:55834574889 2024-09-13T09:20:14.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574888 for osd.6 2024-09-13T09:20:14.920 INFO:teuthology.orchestra.run.smithi032.stdout:55834574889 2024-09-13T09:20:14.920 DEBUG:teuthology.parallel:result is None 2024-09-13T09:20:14.924 INFO:teuthology.orchestra.run.smithi032.stdout:55834574888 2024-09-13T09:20:14.932 INFO:teuthology.orchestra.run.smithi032.stdout:55834574889 2024-09-13T09:20:14.944 INFO:teuthology.orchestra.run.smithi032.stdout:55834574888 2024-09-13T09:20:14.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574889 got 55834574889 for osd.0 2024-09-13T09:20:14.953 DEBUG:teuthology.parallel:result is None 2024-09-13T09:20:14.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574889 got 55834574889 for osd.2 2024-09-13T09:20:14.954 DEBUG:teuthology.parallel:result is None 2024-09-13T09:20:14.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574888 for osd.3 2024-09-13T09:20:14.957 INFO:teuthology.orchestra.run.smithi032.stdout:55834574887 2024-09-13T09:20:14.958 DEBUG:teuthology.parallel:result is None 2024-09-13T09:20:14.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574889 got 55834574889 for osd.4 2024-09-13T09:20:14.960 DEBUG:teuthology.parallel:result is None 2024-09-13T09:20:14.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574888 for osd.7 2024-09-13T09:20:14.975 DEBUG:teuthology.parallel:result is None 2024-09-13T09:20:14.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574887 for osd.1 2024-09-13T09:20:14.976 DEBUG:teuthology.parallel:result is None 2024-09-13T09:20:14.989 INFO:teuthology.orchestra.run.smithi032.stdout:55834574887 2024-09-13T09:20:15.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574887 for osd.5 2024-09-13T09:20:15.006 DEBUG:teuthology.parallel:result is None 2024-09-13T09:20:15.006 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:20:15.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:20:15.331 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:20:15.332 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:20:15.347 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-13T09:20:14.105847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":152,"num_read":127031,"num_read_kb":555824,"num_write":19294,"num_write_kb":381567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1476,"num_bytes_recovered":29874402,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16804,"ondisk_log_size":16804,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":391788,"kb_used_data":15728,"kb_used_omap":12,"kb_used_meta":375923,"kb_avail":754582932,"statfs":{"total":773094113280,"available":772692922368,"internally_reserved":0,"allocated":16105472,"data_stored":8796668,"data_compressed":19717,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12744,"internal_metadata":384945720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.278069"},"pg_stats":[{"pgid":"3.1f","version":"149'406","reported_seq":1883,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816856+0000","last_change":"2024-09-13T09:19:51.725659+0000","last_active":"2024-09-13T09:20:11.816856+0000","last_peered":"2024-09-13T09:20:11.816856+0000","last_clean":"2024-09-13T09:20:11.816856+0000","last_became_active":"2024-09-13T09:17:40.791187+0000","last_became_peered":"2024-09-13T09:17:40.791187+0000","last_unstale":"2024-09-13T09:20:11.816856+0000","last_undegraded":"2024-09-13T09:20:11.816856+0000","last_fullsized":"2024-09-13T09:20:11.816856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:13.280600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001951139,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2539,"num_read_kb":13964,"num_write":479,"num_write_kb":12323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"146'640","reported_seq":3283,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844414+0000","last_change":"2024-09-13T09:19:51.712843+0000","last_active":"2024-09-13T09:20:10.844414+0000","last_peered":"2024-09-13T09:20:10.844414+0000","last_clean":"2024-09-13T09:20:10.844414+0000","last_became_active":"2024-09-13T09:19:20.802379+0000","last_became_peered":"2024-09-13T09:19:20.802379+0000","last_unstale":"2024-09-13T09:20:10.844414+0000","last_undegraded":"2024-09-13T09:20:10.844414+0000","last_fullsized":"2024-09-13T09:20:10.844414+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:20.952086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001400552,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3630,"num_read_kb":10038,"num_write":856,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":8249616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"146'488","reported_seq":3761,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.196764+0000","last_change":"2024-09-13T09:19:51.714853+0000","last_active":"2024-09-13T09:20:10.196764+0000","last_peered":"2024-09-13T09:20:10.196764+0000","last_clean":"2024-09-13T09:20:10.196764+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:20:10.196764+0000","last_undegraded":"2024-09-13T09:20:10.196764+0000","last_fullsized":"2024-09-13T09:20:10.196764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:20.509991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021011329999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5768,"num_read_kb":20052,"num_write":439,"num_write_kb":19543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"149'793","reported_seq":4564,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.196557+0000","last_change":"2024-09-13T09:19:51.715407+0000","last_active":"2024-09-13T09:20:10.196557+0000","last_peered":"2024-09-13T09:20:10.196557+0000","last_clean":"2024-09-13T09:20:10.196557+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:20:10.196557+0000","last_undegraded":"2024-09-13T09:20:10.196557+0000","last_fullsized":"2024-09-13T09:20:10.196557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:05.002231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025684319999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":7779,"num_read_kb":18614,"num_write":881,"num_write_kb":10036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"148'531","reported_seq":4431,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.861804+0000","last_change":"2024-09-13T09:19:51.725489+0000","last_active":"2024-09-13T09:20:11.861804+0000","last_peered":"2024-09-13T09:20:11.861804+0000","last_clean":"2024-09-13T09:20:11.861804+0000","last_became_active":"2024-09-13T09:17:40.799618+0000","last_became_peered":"2024-09-13T09:17:40.799618+0000","last_unstale":"2024-09-13T09:20:11.861804+0000","last_undegraded":"2024-09-13T09:20:11.861804+0000","last_fullsized":"2024-09-13T09:20:11.861804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:00.757881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001823089,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5766,"num_read_kb":19616,"num_write":659,"num_write_kb":9141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":65341,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"146'344","reported_seq":3288,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335538+0000","last_change":"2024-09-13T09:19:51.709691+0000","last_active":"2024-09-13T09:20:12.335538+0000","last_peered":"2024-09-13T09:20:12.335538+0000","last_clean":"2024-09-13T09:20:12.335538+0000","last_became_active":"2024-09-13T09:18:03.973063+0000","last_became_peered":"2024-09-13T09:18:03.973063+0000","last_unstale":"2024-09-13T09:20:12.335538+0000","last_undegraded":"2024-09-13T09:20:12.335538+0000","last_fullsized":"2024-09-13T09:20:12.335538+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:29.911994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082503800000000005,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3197,"num_read_kb":8961,"num_write":344,"num_write_kb":6710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":889520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'715","reported_seq":4249,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.899464+0000","last_change":"2024-09-13T09:19:51.709213+0000","last_active":"2024-09-13T09:20:08.899464+0000","last_peered":"2024-09-13T09:20:08.899464+0000","last_clean":"2024-09-13T09:20:08.899464+0000","last_became_active":"2024-09-13T09:18:06.975016+0000","last_became_peered":"2024-09-13T09:18:06.975016+0000","last_unstale":"2024-09-13T09:20:08.899464+0000","last_undegraded":"2024-09-13T09:20:08.899464+0000","last_fullsized":"2024-09-13T09:20:08.899464+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:30.489567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010502549999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5493,"num_read_kb":28565,"num_write":1011,"num_write_kb":11903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":439303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"149'217","reported_seq":2561,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844055+0000","last_change":"2024-09-13T09:19:51.710337+0000","last_active":"2024-09-13T09:20:10.844055+0000","last_peered":"2024-09-13T09:20:10.844055+0000","last_clean":"2024-09-13T09:20:10.844055+0000","last_became_active":"2024-09-13T09:17:40.797607+0000","last_became_peered":"2024-09-13T09:17:40.797607+0000","last_unstale":"2024-09-13T09:20:10.844055+0000","last_undegraded":"2024-09-13T09:20:10.844055+0000","last_fullsized":"2024-09-13T09:20:10.844055+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:09.025976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011389530000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2065,"num_read_kb":17454,"num_write":181,"num_write_kb":10975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":42856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"148'442","reported_seq":2301,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816853+0000","last_change":"2024-09-13T09:19:51.725896+0000","last_active":"2024-09-13T09:20:11.816853+0000","last_peered":"2024-09-13T09:20:11.816853+0000","last_clean":"2024-09-13T09:20:11.816853+0000","last_became_active":"2024-09-13T09:17:43.809533+0000","last_became_peered":"2024-09-13T09:17:43.809533+0000","last_unstale":"2024-09-13T09:20:11.816853+0000","last_undegraded":"2024-09-13T09:20:11.816853+0000","last_fullsized":"2024-09-13T09:20:11.816853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:20.389195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018671040000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3080,"num_read_kb":24506,"num_write":379,"num_write_kb":12955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":16384,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"148'272","reported_seq":2283,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.196682+0000","last_change":"2024-09-13T09:19:51.714263+0000","last_active":"2024-09-13T09:20:10.196682+0000","last_peered":"2024-09-13T09:20:10.196682+0000","last_clean":"2024-09-13T09:20:10.196682+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:20:10.196682+0000","last_undegraded":"2024-09-13T09:20:10.196682+0000","last_fullsized":"2024-09-13T09:20:10.196682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:29.849557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001510742,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1901,"num_read_kb":15086,"num_write":247,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"147'371","reported_seq":2968,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337807+0000","last_change":"2024-09-13T09:19:51.723974+0000","last_active":"2024-09-13T09:20:12.337807+0000","last_peered":"2024-09-13T09:20:12.337807+0000","last_clean":"2024-09-13T09:20:12.337807+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:20:12.337807+0000","last_undegraded":"2024-09-13T09:20:12.337807+0000","last_fullsized":"2024-09-13T09:20:12.337807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:10.717628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029658509999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3481,"num_read_kb":18288,"num_write":412,"num_write_kb":15933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"142'580","reported_seq":3079,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336038+0000","last_change":"2024-09-13T09:19:51.709951+0000","last_active":"2024-09-13T09:20:12.336038+0000","last_peered":"2024-09-13T09:20:12.336038+0000","last_clean":"2024-09-13T09:20:12.336038+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:20:12.336038+0000","last_undegraded":"2024-09-13T09:20:12.336038+0000","last_fullsized":"2024-09-13T09:20:12.336038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:42.341412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048134,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3471,"num_read_kb":17629,"num_write":764,"num_write_kb":18883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902086+0000","last_change":"2024-09-13T09:19:17.796958+0000","last_active":"2024-09-13T09:20:08.902086+0000","last_peered":"2024-09-13T09:20:08.902086+0000","last_clean":"2024-09-13T09:20:08.902086+0000","last_became_active":"2024-09-13T09:19:17.796177+0000","last_became_peered":"2024-09-13T09:19:17.796177+0000","last_unstale":"2024-09-13T09:20:08.902086+0000","last_undegraded":"2024-09-13T09:20:08.902086+0000","last_fullsized":"2024-09-13T09:20:08.902086+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:59.576027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"149'450","reported_seq":1829,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816376+0000","last_change":"2024-09-13T09:19:51.725161+0000","last_active":"2024-09-13T09:20:11.816376+0000","last_peered":"2024-09-13T09:20:11.816376+0000","last_clean":"2024-09-13T09:20:11.816376+0000","last_became_active":"2024-09-13T09:19:14.748293+0000","last_became_peered":"2024-09-13T09:19:14.748293+0000","last_unstale":"2024-09-13T09:20:11.816376+0000","last_undegraded":"2024-09-13T09:20:11.816376+0000","last_fullsized":"2024-09-13T09:20:11.816376+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:05.560046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016656990000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2578,"num_read_kb":17393,"num_write":357,"num_write_kb":10507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":7471802,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845131+0000","last_change":"2024-09-13T09:19:17.770210+0000","last_active":"2024-09-13T09:20:10.845131+0000","last_peered":"2024-09-13T09:20:10.845131+0000","last_clean":"2024-09-13T09:20:10.845131+0000","last_became_active":"2024-09-13T09:19:17.769570+0000","last_became_peered":"2024-09-13T09:19:17.769570+0000","last_unstale":"2024-09-13T09:20:10.845131+0000","last_undegraded":"2024-09-13T09:20:10.845131+0000","last_fullsized":"2024-09-13T09:20:10.845131+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:37.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"147'561","reported_seq":3012,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845458+0000","last_change":"2024-09-13T09:19:51.712759+0000","last_active":"2024-09-13T09:20:10.845458+0000","last_peered":"2024-09-13T09:20:10.845458+0000","last_clean":"2024-09-13T09:20:10.845458+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:20:10.845458+0000","last_undegraded":"2024-09-13T09:20:10.845458+0000","last_fullsized":"2024-09-13T09:20:10.845458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:51.225828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015869739999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4402,"num_read_kb":8559,"num_write":727,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338299+0000","last_change":"2024-09-13T09:17:40.794662+0000","last_active":"2024-09-13T09:20:12.338299+0000","last_peered":"2024-09-13T09:20:12.338299+0000","last_clean":"2024-09-13T09:20:12.338299+0000","last_became_active":"2024-09-13T09:17:40.793882+0000","last_became_peered":"2024-09-13T09:17:40.793882+0000","last_unstale":"2024-09-13T09:20:12.338299+0000","last_undegraded":"2024-09-13T09:20:12.338299+0000","last_fullsized":"2024-09-13T09:20:12.338299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:22.922753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"148'377","reported_seq":1621,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:09.116120+0000","last_change":"2024-09-13T09:19:51.709972+0000","last_active":"2024-09-13T09:20:09.116120+0000","last_peered":"2024-09-13T09:20:09.116120+0000","last_clean":"2024-09-13T09:20:09.116120+0000","last_became_active":"2024-09-13T09:18:03.973314+0000","last_became_peered":"2024-09-13T09:18:03.973314+0000","last_unstale":"2024-09-13T09:20:09.116120+0000","last_undegraded":"2024-09-13T09:20:09.116120+0000","last_fullsized":"2024-09-13T09:20:09.116120+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.256457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012376119999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1399,"num_read_kb":26203,"num_write":287,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":167476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":120,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:09.116091+0000","last_change":"2024-09-13T09:18:03.968679+0000","last_active":"2024-09-13T09:20:09.116091+0000","last_peered":"2024-09-13T09:20:09.116091+0000","last_clean":"2024-09-13T09:20:09.116091+0000","last_became_active":"2024-09-13T09:18:03.967133+0000","last_became_peered":"2024-09-13T09:18:03.967133+0000","last_unstale":"2024-09-13T09:20:09.116091+0000","last_undegraded":"2024-09-13T09:20:09.116091+0000","last_fullsized":"2024-09-13T09:20:09.116091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:06.643934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"146'344","reported_seq":2267,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.196788+0000","last_change":"2024-09-13T09:19:51.713872+0000","last_active":"2024-09-13T09:20:10.196788+0000","last_peered":"2024-09-13T09:20:10.196788+0000","last_clean":"2024-09-13T09:20:10.196788+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:20:10.196788+0000","last_undegraded":"2024-09-13T09:20:10.196788+0000","last_fullsized":"2024-09-13T09:20:10.196788+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:00.161632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00109892,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2142,"num_read_kb":9020,"num_write":356,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"149'655","reported_seq":4751,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335893+0000","last_change":"2024-09-13T09:19:51.709444+0000","last_active":"2024-09-13T09:20:12.335893+0000","last_peered":"2024-09-13T09:20:12.335893+0000","last_clean":"2024-09-13T09:20:12.335893+0000","last_became_active":"2024-09-13T09:17:40.800027+0000","last_became_peered":"2024-09-13T09:17:40.800027+0000","last_unstale":"2024-09-13T09:20:12.335893+0000","last_undegraded":"2024-09-13T09:20:12.335893+0000","last_fullsized":"2024-09-13T09:20:12.335893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:10.182783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072429099999999995,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":5951,"num_read_kb":14391,"num_write":711,"num_write_kb":6974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":29248,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":198,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.196523+0000","last_change":"2024-09-13T09:18:03.976890+0000","last_active":"2024-09-13T09:20:10.196523+0000","last_peered":"2024-09-13T09:20:10.196523+0000","last_clean":"2024-09-13T09:20:10.196523+0000","last_became_active":"2024-09-13T09:18:03.970835+0000","last_became_peered":"2024-09-13T09:18:03.970835+0000","last_unstale":"2024-09-13T09:20:10.196523+0000","last_undegraded":"2024-09-13T09:20:10.196523+0000","last_fullsized":"2024-09-13T09:20:10.196523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:05.189083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"147'561","reported_seq":2679,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.899503+0000","last_change":"2024-09-13T09:19:51.709196+0000","last_active":"2024-09-13T09:20:08.899503+0000","last_peered":"2024-09-13T09:20:08.899503+0000","last_clean":"2024-09-13T09:20:08.899503+0000","last_became_active":"2024-09-13T09:18:03.970479+0000","last_became_peered":"2024-09-13T09:18:03.970479+0000","last_unstale":"2024-09-13T09:20:08.899503+0000","last_undegraded":"2024-09-13T09:20:08.899503+0000","last_fullsized":"2024-09-13T09:20:08.899503+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:06.229840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001031719,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5538,"num_read_kb":12570,"num_write":635,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1031390,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:09.833063+0000","last_change":"2024-09-13T09:20:02.757363+0000","last_active":"2024-09-13T09:20:09.833063+0000","last_peered":"2024-09-13T09:20:09.833063+0000","last_clean":"2024-09-13T09:20:09.833063+0000","last_became_active":"2024-09-13T09:20:02.756499+0000","last_became_peered":"2024-09-13T09:20:02.756499+0000","last_unstale":"2024-09-13T09:20:09.833063+0000","last_undegraded":"2024-09-13T09:20:09.833063+0000","last_fullsized":"2024-09-13T09:20:09.833063+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:36.829717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"147'483","reported_seq":3446,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816428+0000","last_change":"2024-09-13T09:19:51.724930+0000","last_active":"2024-09-13T09:20:11.816428+0000","last_peered":"2024-09-13T09:20:11.816428+0000","last_clean":"2024-09-13T09:20:11.816428+0000","last_became_active":"2024-09-13T09:17:40.793065+0000","last_became_peered":"2024-09-13T09:17:40.793065+0000","last_unstale":"2024-09-13T09:20:11.816428+0000","last_undegraded":"2024-09-13T09:20:11.816428+0000","last_fullsized":"2024-09-13T09:20:11.816428+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:38.586460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00175362,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4025,"num_read_kb":26901,"num_write":443,"num_write_kb":18254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":178,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336693+0000","last_change":"2024-09-13T09:18:03.968508+0000","last_active":"2024-09-13T09:20:12.336693+0000","last_peered":"2024-09-13T09:20:12.336693+0000","last_clean":"2024-09-13T09:20:12.336693+0000","last_became_active":"2024-09-13T09:18:03.968092+0000","last_became_peered":"2024-09-13T09:18:03.968092+0000","last_unstale":"2024-09-13T09:20:12.336693+0000","last_undegraded":"2024-09-13T09:20:12.336693+0000","last_fullsized":"2024-09-13T09:20:12.336693+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:39.649827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"148'782","reported_seq":2787,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844329+0000","last_change":"2024-09-13T09:19:51.712266+0000","last_active":"2024-09-13T09:20:10.844329+0000","last_peered":"2024-09-13T09:20:10.844329+0000","last_clean":"2024-09-13T09:20:10.844329+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:20:10.844329+0000","last_undegraded":"2024-09-13T09:20:10.844329+0000","last_fullsized":"2024-09-13T09:20:10.844329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:44.094850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012273519999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4598,"num_read_kb":13738,"num_write":1068,"num_write_kb":11812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":161,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.196562+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:20:10.196562+0000","last_peered":"2024-09-13T09:20:10.196562+0000","last_clean":"2024-09-13T09:20:10.196562+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:20:10.196562+0000","last_undegraded":"2024-09-13T09:20:10.196562+0000","last_fullsized":"2024-09-13T09:20:10.196562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":352,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336826+0000","last_change":"2024-09-13T09:17:43.803525+0000","last_active":"2024-09-13T09:20:12.336826+0000","last_peered":"2024-09-13T09:20:12.336826+0000","last_clean":"2024-09-13T09:20:12.336826+0000","last_became_active":"2024-09-13T09:17:43.802603+0000","last_became_peered":"2024-09-13T09:17:43.802603+0000","last_unstale":"2024-09-13T09:20:12.336826+0000","last_undegraded":"2024-09-13T09:20:12.336826+0000","last_fullsized":"2024-09-13T09:20:12.336826+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:07.561959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"147'650","reported_seq":2724,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844330+0000","last_change":"2024-09-13T09:19:51.713229+0000","last_active":"2024-09-13T09:20:10.844330+0000","last_peered":"2024-09-13T09:20:10.844330+0000","last_clean":"2024-09-13T09:20:10.844330+0000","last_became_active":"2024-09-13T09:18:03.974569+0000","last_became_peered":"2024-09-13T09:18:03.974569+0000","last_unstale":"2024-09-13T09:20:10.844330+0000","last_undegraded":"2024-09-13T09:20:10.844330+0000","last_fullsized":"2024-09-13T09:20:10.844330+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:43.479751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036429610000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4126,"num_read_kb":16863,"num_write":738,"num_write_kb":13412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":741496,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"148'539","reported_seq":2812,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335714+0000","last_change":"2024-09-13T09:19:51.709415+0000","last_active":"2024-09-13T09:20:12.335714+0000","last_peered":"2024-09-13T09:20:12.335714+0000","last_clean":"2024-09-13T09:20:12.335714+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:20:12.335714+0000","last_undegraded":"2024-09-13T09:20:12.335714+0000","last_fullsized":"2024-09-13T09:20:12.335714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:06.071033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070054600000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":11592,"num_write":623,"num_write_kb":5754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"148'505","reported_seq":3878,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902417+0000","last_change":"2024-09-13T09:19:51.826968+0000","last_active":"2024-09-13T09:20:08.902417+0000","last_peered":"2024-09-13T09:20:08.902417+0000","last_clean":"2024-09-13T09:20:08.902417+0000","last_became_active":"2024-09-13T09:19:34.891064+0000","last_became_peered":"2024-09-13T09:19:34.891064+0000","last_unstale":"2024-09-13T09:20:08.902417+0000","last_undegraded":"2024-09-13T09:20:08.902417+0000","last_fullsized":"2024-09-13T09:20:08.902417+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:11.410988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078961199999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":3950,"num_read_kb":15857,"num_write":467,"num_write_kb":7750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":4249683,"num_keys_recovered":143,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"149'429","reported_seq":1836,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844608+0000","last_change":"2024-09-13T09:19:51.712307+0000","last_active":"2024-09-13T09:20:10.844608+0000","last_peered":"2024-09-13T09:20:10.844608+0000","last_clean":"2024-09-13T09:20:10.844608+0000","last_became_active":"2024-09-13T09:17:40.801195+0000","last_became_peered":"2024-09-13T09:17:40.801195+0000","last_unstale":"2024-09-13T09:20:10.844608+0000","last_undegraded":"2024-09-13T09:20:10.844608+0000","last_fullsized":"2024-09-13T09:20:10.844608+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:00:07.773428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195911,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2626,"num_read_kb":8389,"num_write":522,"num_write_kb":11747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"149'683","reported_seq":1687,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337358+0000","last_change":"2024-09-13T09:19:51.722579+0000","last_active":"2024-09-13T09:20:12.337358+0000","last_peered":"2024-09-13T09:20:12.337358+0000","last_clean":"2024-09-13T09:20:12.337358+0000","last_became_active":"2024-09-13T09:19:49.736505+0000","last_became_peered":"2024-09-13T09:19:49.736505+0000","last_unstale":"2024-09-13T09:20:12.337358+0000","last_undegraded":"2024-09-13T09:20:12.337358+0000","last_fullsized":"2024-09-13T09:20:12.337358+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:30.617730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00218943,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2689,"num_read_kb":8298,"num_write":946,"num_write_kb":9158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":43,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"147'693","reported_seq":3232,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338076+0000","last_change":"2024-09-13T09:19:51.724802+0000","last_active":"2024-09-13T09:20:12.338076+0000","last_peered":"2024-09-13T09:20:12.338076+0000","last_clean":"2024-09-13T09:20:12.338076+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:20:12.338076+0000","last_undegraded":"2024-09-13T09:20:12.338076+0000","last_fullsized":"2024-09-13T09:20:12.338076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:25.411265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035729350000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4417,"num_read_kb":31654,"num_write":734,"num_write_kb":30325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"146'383","reported_seq":1377,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338412+0000","last_change":"2024-09-13T09:19:51.721061+0000","last_active":"2024-09-13T09:20:12.338412+0000","last_peered":"2024-09-13T09:20:12.338412+0000","last_clean":"2024-09-13T09:20:12.338412+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:20:12.338412+0000","last_undegraded":"2024-09-13T09:20:12.338412+0000","last_fullsized":"2024-09-13T09:20:12.338412+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:31.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068946100000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1301,"num_read_kb":10607,"num_write":406,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"148'980","reported_seq":4053,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338100+0000","last_change":"2024-09-13T09:19:51.722971+0000","last_active":"2024-09-13T09:20:12.338100+0000","last_peered":"2024-09-13T09:20:12.338100+0000","last_clean":"2024-09-13T09:20:12.338100+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:20:12.338100+0000","last_undegraded":"2024-09-13T09:20:12.338100+0000","last_fullsized":"2024-09-13T09:20:12.338100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:56.423075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024735249999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7129,"num_read_kb":22784,"num_write":1446,"num_write_kb":10311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"146'402","reported_seq":2308,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902107+0000","last_change":"2024-09-13T09:19:51.826873+0000","last_active":"2024-09-13T09:20:08.902107+0000","last_peered":"2024-09-13T09:20:08.902107+0000","last_clean":"2024-09-13T09:20:08.902107+0000","last_became_active":"2024-09-13T09:19:20.803495+0000","last_became_peered":"2024-09-13T09:19:20.803495+0000","last_unstale":"2024-09-13T09:20:08.902107+0000","last_undegraded":"2024-09-13T09:20:08.902107+0000","last_fullsized":"2024-09-13T09:20:08.902107+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:18.288761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072899800000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2117,"num_read_kb":10151,"num_write":496,"num_write_kb":12845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5102507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"148'549","reported_seq":3010,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336735+0000","last_change":"2024-09-13T09:19:51.722468+0000","last_active":"2024-09-13T09:20:12.336735+0000","last_peered":"2024-09-13T09:20:12.336735+0000","last_clean":"2024-09-13T09:20:12.336735+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:20:12.336735+0000","last_undegraded":"2024-09-13T09:20:12.336735+0000","last_fullsized":"2024-09-13T09:20:12.336735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:47.572839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020210359999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":14560,"num_write":713,"num_write_kb":15391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"146'438","reported_seq":4935,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816485+0000","last_change":"2024-09-13T09:19:51.724223+0000","last_active":"2024-09-13T09:20:11.816485+0000","last_peered":"2024-09-13T09:20:11.816485+0000","last_clean":"2024-09-13T09:20:11.816485+0000","last_became_active":"2024-09-13T09:17:40.789499+0000","last_became_peered":"2024-09-13T09:17:40.789499+0000","last_unstale":"2024-09-13T09:20:11.816485+0000","last_undegraded":"2024-09-13T09:20:11.816485+0000","last_fullsized":"2024-09-13T09:20:11.816485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:13.395268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010644070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5803,"num_read_kb":44996,"num_write":398,"num_write_kb":17054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"147'421","reported_seq":3620,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337093+0000","last_change":"2024-09-13T09:19:51.723691+0000","last_active":"2024-09-13T09:20:12.337093+0000","last_peered":"2024-09-13T09:20:12.337093+0000","last_clean":"2024-09-13T09:20:12.337093+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:20:12.337093+0000","last_undegraded":"2024-09-13T09:20:12.337093+0000","last_fullsized":"2024-09-13T09:20:12.337093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:07.893848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003077476,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4122,"num_read_kb":18425,"num_write":408,"num_write_kb":9869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":152,"num_read":126915,"num_read_kb":555724,"num_write":19133,"num_write_kb":379149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1469,"num_bytes_recovered":28693609,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16684,"ondisk_log_size":16684,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":30095,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59352,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312488,"statfs":{"total":96636764160,"available":96575987712,"internally_reserved":0,"allocated":2043904,"data_stored":1536508,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":0.94799999999999995,"15min":0.88900000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":11.683,"5min":11.683,"15min":11.683},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.03,"15min":0.98199999999999998},"min":{"1min":0.48099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":11.843999999999999,"5min":11.843999999999999,"15min":11.843999999999999},"last":0.83799999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.86299999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.222,"5min":1.286,"15min":1.286},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.95199999999999996,"15min":0.98399999999999999},"min":{"1min":0.51200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.089,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":1.119}]},{"osd":2,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80000000000000004,"15min":0.80300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.153},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.94899999999999995,"15min":0.95899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.24,"5min":1.385,"15min":1.385},"last":1.0009999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.038,"15min":0.97399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":12.012,"5min":12.012,"15min":12.012},"last":0.873},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.135,"15min":1.075},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":11.776999999999999,"5min":11.776999999999999,"15min":11.776999999999999},"last":1.28}]},{"osd":4,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":1.028,"15min":1.0609999999999999},"min":{"1min":0.44400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2969999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.4590000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.0649999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.073,"15min":1.071},"min":{"1min":0.61199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.3540000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.121,"15min":1.145},"min":{"1min":0.51900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":3.125,"15min":3.125},"last":1.2190000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.1950000000000001,"15min":1.2330000000000001},"min":{"1min":0.59199999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.667,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.109,"15min":1.119},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.498,"5min":1.996,"15min":1.996},"last":1.3100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41272,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330568,"statfs":{"total":96636764160,"available":96594501632,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.438,"5min":0.314,"15min":0.314},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.877},"min":{"1min":0.40899999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3580000000000001,"5min":2.222,"15min":2.222},"last":1.3580000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.83399999999999996,"15min":0.81200000000000006},"min":{"1min":0.50800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.98199999999999998,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.607,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.52}]},{"osd":2,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.65700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4319999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.88},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.98499999999999999,"15min":1.0029999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.613,"5min":2.508,"15min":2.508},"last":0.628}]},{"osd":3,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.66900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.55,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.55},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.2450000000000001,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":1.4670000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.95199999999999996,"15min":0.97399999999999998},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2669999999999999,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.008,"15min":1},"min":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5800000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.5800000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.92300000000000004,"15min":0.90000000000000002},"min":{"1min":0.58999999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.57599999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.8120000000000001,"5min":2.286,"15min":2.286},"last":1.421}]},{"osd":7,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.093,"15min":1.087},"min":{"1min":0.621,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.611,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.079,"15min":1.0489999999999999},"min":{"1min":0.69899999999999995,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.1339999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55220,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316620,"statfs":{"total":96636764160,"available":96580218880,"internally_reserved":0,"allocated":2002944,"data_stored":952001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":6.3970000000000002,"15min":7.3419999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1739999999999999,"5min":132.32900000000001,"15min":132.32900000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.236,"5min":1.335,"15min":1.335},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.86499999999999999,"15min":0.88500000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96399999999999997,"5min":1.946,"15min":1.946},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.998,"15min":1.0209999999999999},"min":{"1min":0.56899999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1579999999999999,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.85099999999999998,"15min":0.84699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1220000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.88800000000000001,"15min":0.89700000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1719999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.0720000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.93799999999999994,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2849999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0009999999999999,"15min":1.006},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4419999999999999,"5min":1.476,"15min":1.476},"last":1.026}]},{"osd":4,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86899999999999999,"15min":0.86499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3140000000000001,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":6.8070000000000004,"15min":7.7770000000000001},"min":{"1min":0.47699999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.544,"5min":132.23599999999999,"15min":132.23599999999999},"last":1.272}]},{"osd":6,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.23},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.016,"15min":1.0329999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3160000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.5820000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.075,"15min":1.075},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.452,"5min":1.655,"15min":1.655},"last":1.181},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1279999999999999,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6379999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.397}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51116,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320724,"statfs":{"total":96636764160,"available":96584421376,"internally_reserved":0,"allocated":1990656,"data_stored":946159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3560000000000001,"5min":1.357,"15min":1.357},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.94599999999999995,"15min":0.94799999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.331}]},{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91400000000000003,"15min":0.92600000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.81100000000000005,"15min":0.81999999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.877},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.006,"5min":0.88,"15min":0.84299999999999997},"min":{"1min":0.56999999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.111}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86899999999999999,"15min":0.85399999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3300000000000001,"5min":1.575,"15min":1.575},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.069,"5min":1.0249999999999999,"15min":0.97799999999999998},"min":{"1min":0.628,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0899999999999999,"5min":4.7389999999999999,"15min":4.7389999999999999},"last":1.0580000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.45400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.3720000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0329999999999999},"min":{"1min":0.53500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.077,"15min":1.0629999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.4119999999999999,"5min":2.4119999999999999,"15min":2.4119999999999999},"last":1.5720000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.034,"15min":1.026},"min":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5609999999999999,"5min":1.597,"15min":1.597},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.125,"15min":1.131},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.2769999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77372,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94294468,"statfs":{"total":96636764160,"available":96557535232,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.313,"5min":0.215,"15min":0.215},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.96399999999999997,"5min":1.625,"15min":1.625},"last":0.90700000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.026},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.28799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.94299999999999995}]},{"osd":2,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.36899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.425,"5min":1.544,"15min":1.544},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.755,"15min":0.76200000000000001},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.749}]},{"osd":4,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89200000000000002,"15min":0.90600000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.25,"5min":1.669,"15min":1.669},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.93300000000000005,"15min":0.94199999999999995},"min":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.296,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.26}]},{"osd":5,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.97199999999999998},"min":{"1min":0.59999999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.5580000000000001,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.002},{"interface":"front","average":{"1min":0.876,"5min":0.92400000000000004,"15min":0.93200000000000005},"min":{"1min":0.63,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3380000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1719999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.628,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.47,"5min":1.748,"15min":1.748},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.024,"5min":1.0860000000000001,"15min":1.0960000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5620000000000001,"5min":2.484,"15min":2.484},"last":1.649}]},{"osd":7,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.998,"15min":1.006},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.385,"5min":1.806,"15min":1.806},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.665,"15min":1.772},"min":{"1min":0.61299999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.3879999999999999,"5min":15.4,"15min":15.4},"last":1.3540000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41272,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330568,"statfs":{"total":96636764160,"available":96594501632,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":0.68500000000000005,"15min":0.61499999999999999},"min":{"1min":0.23499999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":9.7509999999999994,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.71499999999999997,"15min":0.70099999999999996},"min":{"1min":0.28100000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.8490000000000002,"5min":2.8490000000000002,"15min":2.8490000000000002},"last":1.028}]},{"osd":1,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.92800000000000005,"5min":1.23,"15min":1.23},"last":1.056},{"interface":"front","average":{"1min":0.746,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.738,"5min":1.907,"15min":1.907},"last":1.244}]},{"osd":3,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.857,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.98199999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.59599999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.7709999999999999,"5min":1.782,"15min":1.782},"last":1.351},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80000000000000004,"15min":0.80500000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.171,"5min":1.226,"15min":1.226},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.5109999999999999,"5min":1.847,"15min":1.847},"last":1.254},{"interface":"front","average":{"1min":1.004,"5min":1.0229999999999999,"15min":1.026},"min":{"1min":0.61299999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.827,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.4399999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.93200000000000005,"15min":0.93799999999999994},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2709999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.129},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.99399999999999999,"15min":0.999},"min":{"1min":0.54000000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3380000000000001,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.292}]},{"osd":7,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.123,"15min":1.133},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5940000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.173}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":2007040,"data_stored":952020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.70599999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.429,"5min":1.484,"15min":1.484},"last":0.376}]},{"osd":2,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65500000000000003,"15min":0.67100000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.091,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.40400000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.35299999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.212,"5min":1.264,"15min":1.264},"last":0.501},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76000000000000001,"15min":0.76700000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.92900000000000005,"15min":0.95899999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3029999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.98299999999999998,"15min":1.0129999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.22,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.877}]},{"osd":5,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.0760000000000001,"15min":1.04},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":8.3460000000000001,"5min":8.3460000000000001,"15min":8.3460000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.998,"15min":1.022},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2769999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.024,"15min":1.0489999999999999},"min":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.339,"5min":1.647,"15min":1.647},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.012,"15min":1.0309999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.389,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.79000000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.03,"15min":1.052},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.345,"5min":1.629,"15min":1.629},"last":1.095},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.095,"15min":1.119},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.411,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.85899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33104,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338736,"statfs":{"total":96636764160,"available":96602865664,"internally_reserved":0,"allocated":2027520,"data_stored":1536508,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.30599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.61299999999999999,"15min":0.60199999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.123,"5min":1.284,"15min":1.284},"last":1.006}]},{"osd":2,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71899999999999997,"15min":0.71499999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.034}]},{"osd":3,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.42299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":0.872},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.45900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.196}]},{"osd":4,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.876,"15min":0.872},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.038,"15min":0.96799999999999997},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":9.5649999999999995,"5min":9.5649999999999995,"15min":9.5649999999999995},"last":1.073}]},{"osd":5,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.032,"15min":0.95299999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":9.6120000000000001,"5min":9.6120000000000001,"15min":9.6120000000000001},"last":1.347},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.81599999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.401,"5min":1.476,"15min":1.476},"last":1.282}]},{"osd":6,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0840000000000001,"15min":1.0920000000000001},"min":{"1min":0.68000000000000005,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4710000000000001,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.99399999999999999,"15min":0.997},"min":{"1min":0.61099999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.23}]},{"osd":7,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.024,"15min":1.0309999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4430000000000001,"5min":1.617,"15min":1.617},"last":1.397},{"interface":"front","average":{"1min":1.53,"5min":1.1499999999999999,"15min":1.0860000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":9.6440000000000001,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":1.1499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:20:15.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:20:15.651 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:20:15.652 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:20:15.669 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-13T09:20:14.105847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":152,"num_read":127031,"num_read_kb":555824,"num_write":19294,"num_write_kb":381567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1476,"num_bytes_recovered":29874402,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16804,"ondisk_log_size":16804,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":391788,"kb_used_data":15728,"kb_used_omap":12,"kb_used_meta":375923,"kb_avail":754582932,"statfs":{"total":773094113280,"available":772692922368,"internally_reserved":0,"allocated":16105472,"data_stored":8796668,"data_compressed":19717,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12744,"internal_metadata":384945720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.278069"},"pg_stats":[{"pgid":"3.1f","version":"149'406","reported_seq":1883,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816856+0000","last_change":"2024-09-13T09:19:51.725659+0000","last_active":"2024-09-13T09:20:11.816856+0000","last_peered":"2024-09-13T09:20:11.816856+0000","last_clean":"2024-09-13T09:20:11.816856+0000","last_became_active":"2024-09-13T09:17:40.791187+0000","last_became_peered":"2024-09-13T09:17:40.791187+0000","last_unstale":"2024-09-13T09:20:11.816856+0000","last_undegraded":"2024-09-13T09:20:11.816856+0000","last_fullsized":"2024-09-13T09:20:11.816856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:13.280600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001951139,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2539,"num_read_kb":13964,"num_write":479,"num_write_kb":12323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"146'640","reported_seq":3283,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844414+0000","last_change":"2024-09-13T09:19:51.712843+0000","last_active":"2024-09-13T09:20:10.844414+0000","last_peered":"2024-09-13T09:20:10.844414+0000","last_clean":"2024-09-13T09:20:10.844414+0000","last_became_active":"2024-09-13T09:19:20.802379+0000","last_became_peered":"2024-09-13T09:19:20.802379+0000","last_unstale":"2024-09-13T09:20:10.844414+0000","last_undegraded":"2024-09-13T09:20:10.844414+0000","last_fullsized":"2024-09-13T09:20:10.844414+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:20.952086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001400552,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3630,"num_read_kb":10038,"num_write":856,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":8249616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"146'488","reported_seq":3761,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.196764+0000","last_change":"2024-09-13T09:19:51.714853+0000","last_active":"2024-09-13T09:20:10.196764+0000","last_peered":"2024-09-13T09:20:10.196764+0000","last_clean":"2024-09-13T09:20:10.196764+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:20:10.196764+0000","last_undegraded":"2024-09-13T09:20:10.196764+0000","last_fullsized":"2024-09-13T09:20:10.196764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:20.509991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021011329999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5768,"num_read_kb":20052,"num_write":439,"num_write_kb":19543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"149'793","reported_seq":4564,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.196557+0000","last_change":"2024-09-13T09:19:51.715407+0000","last_active":"2024-09-13T09:20:10.196557+0000","last_peered":"2024-09-13T09:20:10.196557+0000","last_clean":"2024-09-13T09:20:10.196557+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:20:10.196557+0000","last_undegraded":"2024-09-13T09:20:10.196557+0000","last_fullsized":"2024-09-13T09:20:10.196557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:05.002231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025684319999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":7779,"num_read_kb":18614,"num_write":881,"num_write_kb":10036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"148'531","reported_seq":4431,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.861804+0000","last_change":"2024-09-13T09:19:51.725489+0000","last_active":"2024-09-13T09:20:11.861804+0000","last_peered":"2024-09-13T09:20:11.861804+0000","last_clean":"2024-09-13T09:20:11.861804+0000","last_became_active":"2024-09-13T09:17:40.799618+0000","last_became_peered":"2024-09-13T09:17:40.799618+0000","last_unstale":"2024-09-13T09:20:11.861804+0000","last_undegraded":"2024-09-13T09:20:11.861804+0000","last_fullsized":"2024-09-13T09:20:11.861804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:00.757881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001823089,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5766,"num_read_kb":19616,"num_write":659,"num_write_kb":9141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":65341,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"146'344","reported_seq":3288,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335538+0000","last_change":"2024-09-13T09:19:51.709691+0000","last_active":"2024-09-13T09:20:12.335538+0000","last_peered":"2024-09-13T09:20:12.335538+0000","last_clean":"2024-09-13T09:20:12.335538+0000","last_became_active":"2024-09-13T09:18:03.973063+0000","last_became_peered":"2024-09-13T09:18:03.973063+0000","last_unstale":"2024-09-13T09:20:12.335538+0000","last_undegraded":"2024-09-13T09:20:12.335538+0000","last_fullsized":"2024-09-13T09:20:12.335538+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:29.911994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082503800000000005,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3197,"num_read_kb":8961,"num_write":344,"num_write_kb":6710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":889520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'715","reported_seq":4249,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.899464+0000","last_change":"2024-09-13T09:19:51.709213+0000","last_active":"2024-09-13T09:20:08.899464+0000","last_peered":"2024-09-13T09:20:08.899464+0000","last_clean":"2024-09-13T09:20:08.899464+0000","last_became_active":"2024-09-13T09:18:06.975016+0000","last_became_peered":"2024-09-13T09:18:06.975016+0000","last_unstale":"2024-09-13T09:20:08.899464+0000","last_undegraded":"2024-09-13T09:20:08.899464+0000","last_fullsized":"2024-09-13T09:20:08.899464+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:30.489567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010502549999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5493,"num_read_kb":28565,"num_write":1011,"num_write_kb":11903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":439303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"149'217","reported_seq":2561,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844055+0000","last_change":"2024-09-13T09:19:51.710337+0000","last_active":"2024-09-13T09:20:10.844055+0000","last_peered":"2024-09-13T09:20:10.844055+0000","last_clean":"2024-09-13T09:20:10.844055+0000","last_became_active":"2024-09-13T09:17:40.797607+0000","last_became_peered":"2024-09-13T09:17:40.797607+0000","last_unstale":"2024-09-13T09:20:10.844055+0000","last_undegraded":"2024-09-13T09:20:10.844055+0000","last_fullsized":"2024-09-13T09:20:10.844055+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:09.025976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011389530000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2065,"num_read_kb":17454,"num_write":181,"num_write_kb":10975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":42856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"148'442","reported_seq":2301,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816853+0000","last_change":"2024-09-13T09:19:51.725896+0000","last_active":"2024-09-13T09:20:11.816853+0000","last_peered":"2024-09-13T09:20:11.816853+0000","last_clean":"2024-09-13T09:20:11.816853+0000","last_became_active":"2024-09-13T09:17:43.809533+0000","last_became_peered":"2024-09-13T09:17:43.809533+0000","last_unstale":"2024-09-13T09:20:11.816853+0000","last_undegraded":"2024-09-13T09:20:11.816853+0000","last_fullsized":"2024-09-13T09:20:11.816853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:20.389195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018671040000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3080,"num_read_kb":24506,"num_write":379,"num_write_kb":12955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":16384,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"148'272","reported_seq":2283,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.196682+0000","last_change":"2024-09-13T09:19:51.714263+0000","last_active":"2024-09-13T09:20:10.196682+0000","last_peered":"2024-09-13T09:20:10.196682+0000","last_clean":"2024-09-13T09:20:10.196682+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:20:10.196682+0000","last_undegraded":"2024-09-13T09:20:10.196682+0000","last_fullsized":"2024-09-13T09:20:10.196682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:29.849557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001510742,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1901,"num_read_kb":15086,"num_write":247,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"147'371","reported_seq":2968,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337807+0000","last_change":"2024-09-13T09:19:51.723974+0000","last_active":"2024-09-13T09:20:12.337807+0000","last_peered":"2024-09-13T09:20:12.337807+0000","last_clean":"2024-09-13T09:20:12.337807+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:20:12.337807+0000","last_undegraded":"2024-09-13T09:20:12.337807+0000","last_fullsized":"2024-09-13T09:20:12.337807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:10.717628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029658509999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3481,"num_read_kb":18288,"num_write":412,"num_write_kb":15933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"142'580","reported_seq":3079,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336038+0000","last_change":"2024-09-13T09:19:51.709951+0000","last_active":"2024-09-13T09:20:12.336038+0000","last_peered":"2024-09-13T09:20:12.336038+0000","last_clean":"2024-09-13T09:20:12.336038+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:20:12.336038+0000","last_undegraded":"2024-09-13T09:20:12.336038+0000","last_fullsized":"2024-09-13T09:20:12.336038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:42.341412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048134,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3471,"num_read_kb":17629,"num_write":764,"num_write_kb":18883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902086+0000","last_change":"2024-09-13T09:19:17.796958+0000","last_active":"2024-09-13T09:20:08.902086+0000","last_peered":"2024-09-13T09:20:08.902086+0000","last_clean":"2024-09-13T09:20:08.902086+0000","last_became_active":"2024-09-13T09:19:17.796177+0000","last_became_peered":"2024-09-13T09:19:17.796177+0000","last_unstale":"2024-09-13T09:20:08.902086+0000","last_undegraded":"2024-09-13T09:20:08.902086+0000","last_fullsized":"2024-09-13T09:20:08.902086+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:59.576027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"149'450","reported_seq":1829,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816376+0000","last_change":"2024-09-13T09:19:51.725161+0000","last_active":"2024-09-13T09:20:11.816376+0000","last_peered":"2024-09-13T09:20:11.816376+0000","last_clean":"2024-09-13T09:20:11.816376+0000","last_became_active":"2024-09-13T09:19:14.748293+0000","last_became_peered":"2024-09-13T09:19:14.748293+0000","last_unstale":"2024-09-13T09:20:11.816376+0000","last_undegraded":"2024-09-13T09:20:11.816376+0000","last_fullsized":"2024-09-13T09:20:11.816376+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:05.560046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016656990000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2578,"num_read_kb":17393,"num_write":357,"num_write_kb":10507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":7471802,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845131+0000","last_change":"2024-09-13T09:19:17.770210+0000","last_active":"2024-09-13T09:20:10.845131+0000","last_peered":"2024-09-13T09:20:10.845131+0000","last_clean":"2024-09-13T09:20:10.845131+0000","last_became_active":"2024-09-13T09:19:17.769570+0000","last_became_peered":"2024-09-13T09:19:17.769570+0000","last_unstale":"2024-09-13T09:20:10.845131+0000","last_undegraded":"2024-09-13T09:20:10.845131+0000","last_fullsized":"2024-09-13T09:20:10.845131+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:37.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"147'561","reported_seq":3012,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845458+0000","last_change":"2024-09-13T09:19:51.712759+0000","last_active":"2024-09-13T09:20:10.845458+0000","last_peered":"2024-09-13T09:20:10.845458+0000","last_clean":"2024-09-13T09:20:10.845458+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:20:10.845458+0000","last_undegraded":"2024-09-13T09:20:10.845458+0000","last_fullsized":"2024-09-13T09:20:10.845458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:51.225828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015869739999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4402,"num_read_kb":8559,"num_write":727,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338299+0000","last_change":"2024-09-13T09:17:40.794662+0000","last_active":"2024-09-13T09:20:12.338299+0000","last_peered":"2024-09-13T09:20:12.338299+0000","last_clean":"2024-09-13T09:20:12.338299+0000","last_became_active":"2024-09-13T09:17:40.793882+0000","last_became_peered":"2024-09-13T09:17:40.793882+0000","last_unstale":"2024-09-13T09:20:12.338299+0000","last_undegraded":"2024-09-13T09:20:12.338299+0000","last_fullsized":"2024-09-13T09:20:12.338299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:22.922753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"148'377","reported_seq":1621,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:09.116120+0000","last_change":"2024-09-13T09:19:51.709972+0000","last_active":"2024-09-13T09:20:09.116120+0000","last_peered":"2024-09-13T09:20:09.116120+0000","last_clean":"2024-09-13T09:20:09.116120+0000","last_became_active":"2024-09-13T09:18:03.973314+0000","last_became_peered":"2024-09-13T09:18:03.973314+0000","last_unstale":"2024-09-13T09:20:09.116120+0000","last_undegraded":"2024-09-13T09:20:09.116120+0000","last_fullsized":"2024-09-13T09:20:09.116120+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.256457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012376119999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1399,"num_read_kb":26203,"num_write":287,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":167476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":120,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:09.116091+0000","last_change":"2024-09-13T09:18:03.968679+0000","last_active":"2024-09-13T09:20:09.116091+0000","last_peered":"2024-09-13T09:20:09.116091+0000","last_clean":"2024-09-13T09:20:09.116091+0000","last_became_active":"2024-09-13T09:18:03.967133+0000","last_became_peered":"2024-09-13T09:18:03.967133+0000","last_unstale":"2024-09-13T09:20:09.116091+0000","last_undegraded":"2024-09-13T09:20:09.116091+0000","last_fullsized":"2024-09-13T09:20:09.116091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:06.643934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"146'344","reported_seq":2267,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.196788+0000","last_change":"2024-09-13T09:19:51.713872+0000","last_active":"2024-09-13T09:20:10.196788+0000","last_peered":"2024-09-13T09:20:10.196788+0000","last_clean":"2024-09-13T09:20:10.196788+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:20:10.196788+0000","last_undegraded":"2024-09-13T09:20:10.196788+0000","last_fullsized":"2024-09-13T09:20:10.196788+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:00.161632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00109892,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2142,"num_read_kb":9020,"num_write":356,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"149'655","reported_seq":4751,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335893+0000","last_change":"2024-09-13T09:19:51.709444+0000","last_active":"2024-09-13T09:20:12.335893+0000","last_peered":"2024-09-13T09:20:12.335893+0000","last_clean":"2024-09-13T09:20:12.335893+0000","last_became_active":"2024-09-13T09:17:40.800027+0000","last_became_peered":"2024-09-13T09:17:40.800027+0000","last_unstale":"2024-09-13T09:20:12.335893+0000","last_undegraded":"2024-09-13T09:20:12.335893+0000","last_fullsized":"2024-09-13T09:20:12.335893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:10.182783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072429099999999995,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":5951,"num_read_kb":14391,"num_write":711,"num_write_kb":6974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":29248,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":198,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.196523+0000","last_change":"2024-09-13T09:18:03.976890+0000","last_active":"2024-09-13T09:20:10.196523+0000","last_peered":"2024-09-13T09:20:10.196523+0000","last_clean":"2024-09-13T09:20:10.196523+0000","last_became_active":"2024-09-13T09:18:03.970835+0000","last_became_peered":"2024-09-13T09:18:03.970835+0000","last_unstale":"2024-09-13T09:20:10.196523+0000","last_undegraded":"2024-09-13T09:20:10.196523+0000","last_fullsized":"2024-09-13T09:20:10.196523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:05.189083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"147'561","reported_seq":2679,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.899503+0000","last_change":"2024-09-13T09:19:51.709196+0000","last_active":"2024-09-13T09:20:08.899503+0000","last_peered":"2024-09-13T09:20:08.899503+0000","last_clean":"2024-09-13T09:20:08.899503+0000","last_became_active":"2024-09-13T09:18:03.970479+0000","last_became_peered":"2024-09-13T09:18:03.970479+0000","last_unstale":"2024-09-13T09:20:08.899503+0000","last_undegraded":"2024-09-13T09:20:08.899503+0000","last_fullsized":"2024-09-13T09:20:08.899503+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:06.229840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001031719,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5538,"num_read_kb":12570,"num_write":635,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1031390,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:09.833063+0000","last_change":"2024-09-13T09:20:02.757363+0000","last_active":"2024-09-13T09:20:09.833063+0000","last_peered":"2024-09-13T09:20:09.833063+0000","last_clean":"2024-09-13T09:20:09.833063+0000","last_became_active":"2024-09-13T09:20:02.756499+0000","last_became_peered":"2024-09-13T09:20:02.756499+0000","last_unstale":"2024-09-13T09:20:09.833063+0000","last_undegraded":"2024-09-13T09:20:09.833063+0000","last_fullsized":"2024-09-13T09:20:09.833063+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:36.829717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"147'483","reported_seq":3446,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816428+0000","last_change":"2024-09-13T09:19:51.724930+0000","last_active":"2024-09-13T09:20:11.816428+0000","last_peered":"2024-09-13T09:20:11.816428+0000","last_clean":"2024-09-13T09:20:11.816428+0000","last_became_active":"2024-09-13T09:17:40.793065+0000","last_became_peered":"2024-09-13T09:17:40.793065+0000","last_unstale":"2024-09-13T09:20:11.816428+0000","last_undegraded":"2024-09-13T09:20:11.816428+0000","last_fullsized":"2024-09-13T09:20:11.816428+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:38.586460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00175362,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4025,"num_read_kb":26901,"num_write":443,"num_write_kb":18254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":178,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336693+0000","last_change":"2024-09-13T09:18:03.968508+0000","last_active":"2024-09-13T09:20:12.336693+0000","last_peered":"2024-09-13T09:20:12.336693+0000","last_clean":"2024-09-13T09:20:12.336693+0000","last_became_active":"2024-09-13T09:18:03.968092+0000","last_became_peered":"2024-09-13T09:18:03.968092+0000","last_unstale":"2024-09-13T09:20:12.336693+0000","last_undegraded":"2024-09-13T09:20:12.336693+0000","last_fullsized":"2024-09-13T09:20:12.336693+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:39.649827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"148'782","reported_seq":2787,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844329+0000","last_change":"2024-09-13T09:19:51.712266+0000","last_active":"2024-09-13T09:20:10.844329+0000","last_peered":"2024-09-13T09:20:10.844329+0000","last_clean":"2024-09-13T09:20:10.844329+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:20:10.844329+0000","last_undegraded":"2024-09-13T09:20:10.844329+0000","last_fullsized":"2024-09-13T09:20:10.844329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:44.094850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012273519999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4598,"num_read_kb":13738,"num_write":1068,"num_write_kb":11812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":161,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.196562+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:20:10.196562+0000","last_peered":"2024-09-13T09:20:10.196562+0000","last_clean":"2024-09-13T09:20:10.196562+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:20:10.196562+0000","last_undegraded":"2024-09-13T09:20:10.196562+0000","last_fullsized":"2024-09-13T09:20:10.196562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":352,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336826+0000","last_change":"2024-09-13T09:17:43.803525+0000","last_active":"2024-09-13T09:20:12.336826+0000","last_peered":"2024-09-13T09:20:12.336826+0000","last_clean":"2024-09-13T09:20:12.336826+0000","last_became_active":"2024-09-13T09:17:43.802603+0000","last_became_peered":"2024-09-13T09:17:43.802603+0000","last_unstale":"2024-09-13T09:20:12.336826+0000","last_undegraded":"2024-09-13T09:20:12.336826+0000","last_fullsized":"2024-09-13T09:20:12.336826+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:07.561959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"147'650","reported_seq":2724,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844330+0000","last_change":"2024-09-13T09:19:51.713229+0000","last_active":"2024-09-13T09:20:10.844330+0000","last_peered":"2024-09-13T09:20:10.844330+0000","last_clean":"2024-09-13T09:20:10.844330+0000","last_became_active":"2024-09-13T09:18:03.974569+0000","last_became_peered":"2024-09-13T09:18:03.974569+0000","last_unstale":"2024-09-13T09:20:10.844330+0000","last_undegraded":"2024-09-13T09:20:10.844330+0000","last_fullsized":"2024-09-13T09:20:10.844330+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:43.479751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036429610000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4126,"num_read_kb":16863,"num_write":738,"num_write_kb":13412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":741496,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"148'539","reported_seq":2812,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335714+0000","last_change":"2024-09-13T09:19:51.709415+0000","last_active":"2024-09-13T09:20:12.335714+0000","last_peered":"2024-09-13T09:20:12.335714+0000","last_clean":"2024-09-13T09:20:12.335714+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:20:12.335714+0000","last_undegraded":"2024-09-13T09:20:12.335714+0000","last_fullsized":"2024-09-13T09:20:12.335714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:06.071033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070054600000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":11592,"num_write":623,"num_write_kb":5754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"148'505","reported_seq":3878,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902417+0000","last_change":"2024-09-13T09:19:51.826968+0000","last_active":"2024-09-13T09:20:08.902417+0000","last_peered":"2024-09-13T09:20:08.902417+0000","last_clean":"2024-09-13T09:20:08.902417+0000","last_became_active":"2024-09-13T09:19:34.891064+0000","last_became_peered":"2024-09-13T09:19:34.891064+0000","last_unstale":"2024-09-13T09:20:08.902417+0000","last_undegraded":"2024-09-13T09:20:08.902417+0000","last_fullsized":"2024-09-13T09:20:08.902417+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:11.410988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078961199999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":3950,"num_read_kb":15857,"num_write":467,"num_write_kb":7750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":4249683,"num_keys_recovered":143,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"149'429","reported_seq":1836,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844608+0000","last_change":"2024-09-13T09:19:51.712307+0000","last_active":"2024-09-13T09:20:10.844608+0000","last_peered":"2024-09-13T09:20:10.844608+0000","last_clean":"2024-09-13T09:20:10.844608+0000","last_became_active":"2024-09-13T09:17:40.801195+0000","last_became_peered":"2024-09-13T09:17:40.801195+0000","last_unstale":"2024-09-13T09:20:10.844608+0000","last_undegraded":"2024-09-13T09:20:10.844608+0000","last_fullsized":"2024-09-13T09:20:10.844608+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:00:07.773428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195911,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2626,"num_read_kb":8389,"num_write":522,"num_write_kb":11747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"149'683","reported_seq":1687,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337358+0000","last_change":"2024-09-13T09:19:51.722579+0000","last_active":"2024-09-13T09:20:12.337358+0000","last_peered":"2024-09-13T09:20:12.337358+0000","last_clean":"2024-09-13T09:20:12.337358+0000","last_became_active":"2024-09-13T09:19:49.736505+0000","last_became_peered":"2024-09-13T09:19:49.736505+0000","last_unstale":"2024-09-13T09:20:12.337358+0000","last_undegraded":"2024-09-13T09:20:12.337358+0000","last_fullsized":"2024-09-13T09:20:12.337358+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:30.617730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00218943,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2689,"num_read_kb":8298,"num_write":946,"num_write_kb":9158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":43,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"147'693","reported_seq":3232,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338076+0000","last_change":"2024-09-13T09:19:51.724802+0000","last_active":"2024-09-13T09:20:12.338076+0000","last_peered":"2024-09-13T09:20:12.338076+0000","last_clean":"2024-09-13T09:20:12.338076+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:20:12.338076+0000","last_undegraded":"2024-09-13T09:20:12.338076+0000","last_fullsized":"2024-09-13T09:20:12.338076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:25.411265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035729350000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4417,"num_read_kb":31654,"num_write":734,"num_write_kb":30325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"146'383","reported_seq":1377,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338412+0000","last_change":"2024-09-13T09:19:51.721061+0000","last_active":"2024-09-13T09:20:12.338412+0000","last_peered":"2024-09-13T09:20:12.338412+0000","last_clean":"2024-09-13T09:20:12.338412+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:20:12.338412+0000","last_undegraded":"2024-09-13T09:20:12.338412+0000","last_fullsized":"2024-09-13T09:20:12.338412+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:31.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068946100000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1301,"num_read_kb":10607,"num_write":406,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"148'980","reported_seq":4053,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338100+0000","last_change":"2024-09-13T09:19:51.722971+0000","last_active":"2024-09-13T09:20:12.338100+0000","last_peered":"2024-09-13T09:20:12.338100+0000","last_clean":"2024-09-13T09:20:12.338100+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:20:12.338100+0000","last_undegraded":"2024-09-13T09:20:12.338100+0000","last_fullsized":"2024-09-13T09:20:12.338100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:56.423075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024735249999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7129,"num_read_kb":22784,"num_write":1446,"num_write_kb":10311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"146'402","reported_seq":2308,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902107+0000","last_change":"2024-09-13T09:19:51.826873+0000","last_active":"2024-09-13T09:20:08.902107+0000","last_peered":"2024-09-13T09:20:08.902107+0000","last_clean":"2024-09-13T09:20:08.902107+0000","last_became_active":"2024-09-13T09:19:20.803495+0000","last_became_peered":"2024-09-13T09:19:20.803495+0000","last_unstale":"2024-09-13T09:20:08.902107+0000","last_undegraded":"2024-09-13T09:20:08.902107+0000","last_fullsized":"2024-09-13T09:20:08.902107+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:18.288761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072899800000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2117,"num_read_kb":10151,"num_write":496,"num_write_kb":12845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5102507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"148'549","reported_seq":3010,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336735+0000","last_change":"2024-09-13T09:19:51.722468+0000","last_active":"2024-09-13T09:20:12.336735+0000","last_peered":"2024-09-13T09:20:12.336735+0000","last_clean":"2024-09-13T09:20:12.336735+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:20:12.336735+0000","last_undegraded":"2024-09-13T09:20:12.336735+0000","last_fullsized":"2024-09-13T09:20:12.336735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:47.572839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020210359999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":14560,"num_write":713,"num_write_kb":15391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"146'438","reported_seq":4935,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816485+0000","last_change":"2024-09-13T09:19:51.724223+0000","last_active":"2024-09-13T09:20:11.816485+0000","last_peered":"2024-09-13T09:20:11.816485+0000","last_clean":"2024-09-13T09:20:11.816485+0000","last_became_active":"2024-09-13T09:17:40.789499+0000","last_became_peered":"2024-09-13T09:17:40.789499+0000","last_unstale":"2024-09-13T09:20:11.816485+0000","last_undegraded":"2024-09-13T09:20:11.816485+0000","last_fullsized":"2024-09-13T09:20:11.816485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:13.395268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010644070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5803,"num_read_kb":44996,"num_write":398,"num_write_kb":17054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"147'421","reported_seq":3620,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337093+0000","last_change":"2024-09-13T09:19:51.723691+0000","last_active":"2024-09-13T09:20:12.337093+0000","last_peered":"2024-09-13T09:20:12.337093+0000","last_clean":"2024-09-13T09:20:12.337093+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:20:12.337093+0000","last_undegraded":"2024-09-13T09:20:12.337093+0000","last_fullsized":"2024-09-13T09:20:12.337093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:07.893848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003077476,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4122,"num_read_kb":18425,"num_write":408,"num_write_kb":9869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":152,"num_read":126915,"num_read_kb":555724,"num_write":19133,"num_write_kb":379149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1469,"num_bytes_recovered":28693609,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16684,"ondisk_log_size":16684,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":30095,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59352,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312488,"statfs":{"total":96636764160,"available":96575987712,"internally_reserved":0,"allocated":2043904,"data_stored":1536508,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":0.94799999999999995,"15min":0.88900000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":11.683,"5min":11.683,"15min":11.683},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.03,"15min":0.98199999999999998},"min":{"1min":0.48099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":11.843999999999999,"5min":11.843999999999999,"15min":11.843999999999999},"last":0.83799999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.86299999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.222,"5min":1.286,"15min":1.286},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.95199999999999996,"15min":0.98399999999999999},"min":{"1min":0.51200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.089,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":1.119}]},{"osd":2,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80000000000000004,"15min":0.80300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.153},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.94899999999999995,"15min":0.95899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.24,"5min":1.385,"15min":1.385},"last":1.0009999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.038,"15min":0.97399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":12.012,"5min":12.012,"15min":12.012},"last":0.873},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.135,"15min":1.075},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":11.776999999999999,"5min":11.776999999999999,"15min":11.776999999999999},"last":1.28}]},{"osd":4,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":1.028,"15min":1.0609999999999999},"min":{"1min":0.44400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2969999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.4590000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.0649999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.073,"15min":1.071},"min":{"1min":0.61199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.3540000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.121,"15min":1.145},"min":{"1min":0.51900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":3.125,"15min":3.125},"last":1.2190000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.1950000000000001,"15min":1.2330000000000001},"min":{"1min":0.59199999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.667,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.109,"15min":1.119},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.498,"5min":1.996,"15min":1.996},"last":1.3100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41272,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330568,"statfs":{"total":96636764160,"available":96594501632,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.438,"5min":0.314,"15min":0.314},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.877},"min":{"1min":0.40899999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3580000000000001,"5min":2.222,"15min":2.222},"last":1.3580000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.83399999999999996,"15min":0.81200000000000006},"min":{"1min":0.50800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.98199999999999998,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.607,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.52}]},{"osd":2,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.65700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4319999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.88},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.98499999999999999,"15min":1.0029999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.613,"5min":2.508,"15min":2.508},"last":0.628}]},{"osd":3,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.66900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.55,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.55},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.2450000000000001,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":1.4670000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.95199999999999996,"15min":0.97399999999999998},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2669999999999999,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.008,"15min":1},"min":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5800000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.5800000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.92300000000000004,"15min":0.90000000000000002},"min":{"1min":0.58999999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.57599999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.8120000000000001,"5min":2.286,"15min":2.286},"last":1.421}]},{"osd":7,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.093,"15min":1.087},"min":{"1min":0.621,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.611,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.079,"15min":1.0489999999999999},"min":{"1min":0.69899999999999995,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.1339999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55220,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316620,"statfs":{"total":96636764160,"available":96580218880,"internally_reserved":0,"allocated":2002944,"data_stored":952001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":6.3970000000000002,"15min":7.3419999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1739999999999999,"5min":132.32900000000001,"15min":132.32900000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.236,"5min":1.335,"15min":1.335},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.86499999999999999,"15min":0.88500000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96399999999999997,"5min":1.946,"15min":1.946},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.998,"15min":1.0209999999999999},"min":{"1min":0.56899999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1579999999999999,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.85099999999999998,"15min":0.84699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1220000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.88800000000000001,"15min":0.89700000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1719999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.0720000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.93799999999999994,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2849999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0009999999999999,"15min":1.006},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4419999999999999,"5min":1.476,"15min":1.476},"last":1.026}]},{"osd":4,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86899999999999999,"15min":0.86499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3140000000000001,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":6.8070000000000004,"15min":7.7770000000000001},"min":{"1min":0.47699999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.544,"5min":132.23599999999999,"15min":132.23599999999999},"last":1.272}]},{"osd":6,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.23},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.016,"15min":1.0329999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3160000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.5820000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.075,"15min":1.075},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.452,"5min":1.655,"15min":1.655},"last":1.181},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1279999999999999,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6379999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.397}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51116,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320724,"statfs":{"total":96636764160,"available":96584421376,"internally_reserved":0,"allocated":1990656,"data_stored":946159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3560000000000001,"5min":1.357,"15min":1.357},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.94599999999999995,"15min":0.94799999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.331}]},{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91400000000000003,"15min":0.92600000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.81100000000000005,"15min":0.81999999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.877},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.006,"5min":0.88,"15min":0.84299999999999997},"min":{"1min":0.56999999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.111}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86899999999999999,"15min":0.85399999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3300000000000001,"5min":1.575,"15min":1.575},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.069,"5min":1.0249999999999999,"15min":0.97799999999999998},"min":{"1min":0.628,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0899999999999999,"5min":4.7389999999999999,"15min":4.7389999999999999},"last":1.0580000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.45400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.3720000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0329999999999999},"min":{"1min":0.53500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.077,"15min":1.0629999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.4119999999999999,"5min":2.4119999999999999,"15min":2.4119999999999999},"last":1.5720000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.034,"15min":1.026},"min":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5609999999999999,"5min":1.597,"15min":1.597},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.125,"15min":1.131},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.2769999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77372,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94294468,"statfs":{"total":96636764160,"available":96557535232,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.313,"5min":0.215,"15min":0.215},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.96399999999999997,"5min":1.625,"15min":1.625},"last":0.90700000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.026},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.28799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.94299999999999995}]},{"osd":2,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.36899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.425,"5min":1.544,"15min":1.544},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.755,"15min":0.76200000000000001},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.749}]},{"osd":4,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89200000000000002,"15min":0.90600000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.25,"5min":1.669,"15min":1.669},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.93300000000000005,"15min":0.94199999999999995},"min":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.296,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.26}]},{"osd":5,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.97199999999999998},"min":{"1min":0.59999999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.5580000000000001,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.002},{"interface":"front","average":{"1min":0.876,"5min":0.92400000000000004,"15min":0.93200000000000005},"min":{"1min":0.63,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3380000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1719999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.628,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.47,"5min":1.748,"15min":1.748},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.024,"5min":1.0860000000000001,"15min":1.0960000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5620000000000001,"5min":2.484,"15min":2.484},"last":1.649}]},{"osd":7,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.998,"15min":1.006},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.385,"5min":1.806,"15min":1.806},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.665,"15min":1.772},"min":{"1min":0.61299999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.3879999999999999,"5min":15.4,"15min":15.4},"last":1.3540000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41272,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330568,"statfs":{"total":96636764160,"available":96594501632,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":0.68500000000000005,"15min":0.61499999999999999},"min":{"1min":0.23499999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":9.7509999999999994,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.71499999999999997,"15min":0.70099999999999996},"min":{"1min":0.28100000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.8490000000000002,"5min":2.8490000000000002,"15min":2.8490000000000002},"last":1.028}]},{"osd":1,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.92800000000000005,"5min":1.23,"15min":1.23},"last":1.056},{"interface":"front","average":{"1min":0.746,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.738,"5min":1.907,"15min":1.907},"last":1.244}]},{"osd":3,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.857,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.98199999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.59599999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.7709999999999999,"5min":1.782,"15min":1.782},"last":1.351},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80000000000000004,"15min":0.80500000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.171,"5min":1.226,"15min":1.226},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.5109999999999999,"5min":1.847,"15min":1.847},"last":1.254},{"interface":"front","average":{"1min":1.004,"5min":1.0229999999999999,"15min":1.026},"min":{"1min":0.61299999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.827,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.4399999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.93200000000000005,"15min":0.93799999999999994},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2709999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.129},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.99399999999999999,"15min":0.999},"min":{"1min":0.54000000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3380000000000001,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.292}]},{"osd":7,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.123,"15min":1.133},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5940000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.173}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":2007040,"data_stored":952020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.70599999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.429,"5min":1.484,"15min":1.484},"last":0.376}]},{"osd":2,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65500000000000003,"15min":0.67100000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.091,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.40400000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.35299999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.212,"5min":1.264,"15min":1.264},"last":0.501},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76000000000000001,"15min":0.76700000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.92900000000000005,"15min":0.95899999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3029999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.98299999999999998,"15min":1.0129999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.22,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.877}]},{"osd":5,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.0760000000000001,"15min":1.04},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":8.3460000000000001,"5min":8.3460000000000001,"15min":8.3460000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.998,"15min":1.022},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2769999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.024,"15min":1.0489999999999999},"min":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.339,"5min":1.647,"15min":1.647},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.012,"15min":1.0309999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.389,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.79000000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.03,"15min":1.052},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.345,"5min":1.629,"15min":1.629},"last":1.095},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.095,"15min":1.119},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.411,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.85899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33104,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338736,"statfs":{"total":96636764160,"available":96602865664,"internally_reserved":0,"allocated":2027520,"data_stored":1536508,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.30599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.61299999999999999,"15min":0.60199999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.123,"5min":1.284,"15min":1.284},"last":1.006}]},{"osd":2,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71899999999999997,"15min":0.71499999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.034}]},{"osd":3,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.42299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":0.872},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.45900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.196}]},{"osd":4,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.876,"15min":0.872},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.038,"15min":0.96799999999999997},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":9.5649999999999995,"5min":9.5649999999999995,"15min":9.5649999999999995},"last":1.073}]},{"osd":5,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.032,"15min":0.95299999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":9.6120000000000001,"5min":9.6120000000000001,"15min":9.6120000000000001},"last":1.347},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.81599999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.401,"5min":1.476,"15min":1.476},"last":1.282}]},{"osd":6,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0840000000000001,"15min":1.0920000000000001},"min":{"1min":0.68000000000000005,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4710000000000001,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.99399999999999999,"15min":0.997},"min":{"1min":0.61099999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.23}]},{"osd":7,"last update":"Fri Sep 13 09:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.024,"15min":1.0309999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4430000000000001,"5min":1.617,"15min":1.617},"last":1.397},{"interface":"front","average":{"1min":1.53,"5min":1.1499999999999999,"15min":1.0860000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":9.6440000000000001,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":1.1499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:20:15.672 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:20:15.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-13T09:20:15.977 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T09:20:15.987 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T09:20:15.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:20:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:20:16.277 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:20:16.293 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-13T09:20:16.106477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":152,"num_read":127031,"num_read_kb":555824,"num_write":19294,"num_write_kb":381567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1476,"num_bytes_recovered":29874402,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16804,"ondisk_log_size":16804,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":391880,"kb_used_data":15824,"kb_used_omap":12,"kb_used_meta":375923,"kb_avail":754582840,"statfs":{"total":773094113280,"available":772692828160,"internally_reserved":0,"allocated":16203776,"data_stored":8843404,"data_compressed":19717,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12744,"internal_metadata":384945720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.276048"},"pg_stats":[{"pgid":"3.1f","version":"149'406","reported_seq":1883,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816856+0000","last_change":"2024-09-13T09:19:51.725659+0000","last_active":"2024-09-13T09:20:11.816856+0000","last_peered":"2024-09-13T09:20:11.816856+0000","last_clean":"2024-09-13T09:20:11.816856+0000","last_became_active":"2024-09-13T09:17:40.791187+0000","last_became_peered":"2024-09-13T09:17:40.791187+0000","last_unstale":"2024-09-13T09:20:11.816856+0000","last_undegraded":"2024-09-13T09:20:11.816856+0000","last_fullsized":"2024-09-13T09:20:11.816856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:13.280600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001951139,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2539,"num_read_kb":13964,"num_write":479,"num_write_kb":12323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"146'640","reported_seq":3283,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844414+0000","last_change":"2024-09-13T09:19:51.712843+0000","last_active":"2024-09-13T09:20:10.844414+0000","last_peered":"2024-09-13T09:20:10.844414+0000","last_clean":"2024-09-13T09:20:10.844414+0000","last_became_active":"2024-09-13T09:19:20.802379+0000","last_became_peered":"2024-09-13T09:19:20.802379+0000","last_unstale":"2024-09-13T09:20:10.844414+0000","last_undegraded":"2024-09-13T09:20:10.844414+0000","last_fullsized":"2024-09-13T09:20:10.844414+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:20.952086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001400552,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3630,"num_read_kb":10038,"num_write":856,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":8249616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"146'488","reported_seq":3762,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844008+0000","last_change":"2024-09-13T09:19:51.714853+0000","last_active":"2024-09-13T09:20:10.844008+0000","last_peered":"2024-09-13T09:20:10.844008+0000","last_clean":"2024-09-13T09:20:10.844008+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:20:10.844008+0000","last_undegraded":"2024-09-13T09:20:10.844008+0000","last_fullsized":"2024-09-13T09:20:10.844008+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:20.509991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021011329999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5768,"num_read_kb":20052,"num_write":439,"num_write_kb":19543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"149'793","reported_seq":4565,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.843864+0000","last_change":"2024-09-13T09:19:51.715407+0000","last_active":"2024-09-13T09:20:10.843864+0000","last_peered":"2024-09-13T09:20:10.843864+0000","last_clean":"2024-09-13T09:20:10.843864+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:20:10.843864+0000","last_undegraded":"2024-09-13T09:20:10.843864+0000","last_fullsized":"2024-09-13T09:20:10.843864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:05.002231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025684319999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":7779,"num_read_kb":18614,"num_write":881,"num_write_kb":10036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"148'531","reported_seq":4431,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.861804+0000","last_change":"2024-09-13T09:19:51.725489+0000","last_active":"2024-09-13T09:20:11.861804+0000","last_peered":"2024-09-13T09:20:11.861804+0000","last_clean":"2024-09-13T09:20:11.861804+0000","last_became_active":"2024-09-13T09:17:40.799618+0000","last_became_peered":"2024-09-13T09:17:40.799618+0000","last_unstale":"2024-09-13T09:20:11.861804+0000","last_undegraded":"2024-09-13T09:20:11.861804+0000","last_fullsized":"2024-09-13T09:20:11.861804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:00.757881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001823089,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5766,"num_read_kb":19616,"num_write":659,"num_write_kb":9141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":65341,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"146'344","reported_seq":3288,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335538+0000","last_change":"2024-09-13T09:19:51.709691+0000","last_active":"2024-09-13T09:20:12.335538+0000","last_peered":"2024-09-13T09:20:12.335538+0000","last_clean":"2024-09-13T09:20:12.335538+0000","last_became_active":"2024-09-13T09:18:03.973063+0000","last_became_peered":"2024-09-13T09:18:03.973063+0000","last_unstale":"2024-09-13T09:20:12.335538+0000","last_undegraded":"2024-09-13T09:20:12.335538+0000","last_fullsized":"2024-09-13T09:20:12.335538+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:29.911994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082503800000000005,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3197,"num_read_kb":8961,"num_write":344,"num_write_kb":6710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":889520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'715","reported_seq":4250,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844995+0000","last_change":"2024-09-13T09:19:51.709213+0000","last_active":"2024-09-13T09:20:10.844995+0000","last_peered":"2024-09-13T09:20:10.844995+0000","last_clean":"2024-09-13T09:20:10.844995+0000","last_became_active":"2024-09-13T09:18:06.975016+0000","last_became_peered":"2024-09-13T09:18:06.975016+0000","last_unstale":"2024-09-13T09:20:10.844995+0000","last_undegraded":"2024-09-13T09:20:10.844995+0000","last_fullsized":"2024-09-13T09:20:10.844995+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:30.489567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010502549999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5493,"num_read_kb":28565,"num_write":1011,"num_write_kb":11903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":439303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"149'217","reported_seq":2561,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844055+0000","last_change":"2024-09-13T09:19:51.710337+0000","last_active":"2024-09-13T09:20:10.844055+0000","last_peered":"2024-09-13T09:20:10.844055+0000","last_clean":"2024-09-13T09:20:10.844055+0000","last_became_active":"2024-09-13T09:17:40.797607+0000","last_became_peered":"2024-09-13T09:17:40.797607+0000","last_unstale":"2024-09-13T09:20:10.844055+0000","last_undegraded":"2024-09-13T09:20:10.844055+0000","last_fullsized":"2024-09-13T09:20:10.844055+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:09.025976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011389530000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2065,"num_read_kb":17454,"num_write":181,"num_write_kb":10975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":42856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"148'442","reported_seq":2301,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816853+0000","last_change":"2024-09-13T09:19:51.725896+0000","last_active":"2024-09-13T09:20:11.816853+0000","last_peered":"2024-09-13T09:20:11.816853+0000","last_clean":"2024-09-13T09:20:11.816853+0000","last_became_active":"2024-09-13T09:17:43.809533+0000","last_became_peered":"2024-09-13T09:17:43.809533+0000","last_unstale":"2024-09-13T09:20:11.816853+0000","last_undegraded":"2024-09-13T09:20:11.816853+0000","last_fullsized":"2024-09-13T09:20:11.816853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:20.389195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018671040000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3080,"num_read_kb":24506,"num_write":379,"num_write_kb":12955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":16384,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"148'272","reported_seq":2284,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844180+0000","last_change":"2024-09-13T09:19:51.714263+0000","last_active":"2024-09-13T09:20:10.844180+0000","last_peered":"2024-09-13T09:20:10.844180+0000","last_clean":"2024-09-13T09:20:10.844180+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:20:10.844180+0000","last_undegraded":"2024-09-13T09:20:10.844180+0000","last_fullsized":"2024-09-13T09:20:10.844180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:29.849557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001510742,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1901,"num_read_kb":15086,"num_write":247,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"147'371","reported_seq":2968,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337807+0000","last_change":"2024-09-13T09:19:51.723974+0000","last_active":"2024-09-13T09:20:12.337807+0000","last_peered":"2024-09-13T09:20:12.337807+0000","last_clean":"2024-09-13T09:20:12.337807+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:20:12.337807+0000","last_undegraded":"2024-09-13T09:20:12.337807+0000","last_fullsized":"2024-09-13T09:20:12.337807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:10.717628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029658509999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3481,"num_read_kb":18288,"num_write":412,"num_write_kb":15933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"142'580","reported_seq":3079,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336038+0000","last_change":"2024-09-13T09:19:51.709951+0000","last_active":"2024-09-13T09:20:12.336038+0000","last_peered":"2024-09-13T09:20:12.336038+0000","last_clean":"2024-09-13T09:20:12.336038+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:20:12.336038+0000","last_undegraded":"2024-09-13T09:20:12.336038+0000","last_fullsized":"2024-09-13T09:20:12.336038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:42.341412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048134,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3471,"num_read_kb":17629,"num_write":764,"num_write_kb":18883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902086+0000","last_change":"2024-09-13T09:19:17.796958+0000","last_active":"2024-09-13T09:20:08.902086+0000","last_peered":"2024-09-13T09:20:08.902086+0000","last_clean":"2024-09-13T09:20:08.902086+0000","last_became_active":"2024-09-13T09:19:17.796177+0000","last_became_peered":"2024-09-13T09:19:17.796177+0000","last_unstale":"2024-09-13T09:20:08.902086+0000","last_undegraded":"2024-09-13T09:20:08.902086+0000","last_fullsized":"2024-09-13T09:20:08.902086+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:59.576027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"149'450","reported_seq":1829,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816376+0000","last_change":"2024-09-13T09:19:51.725161+0000","last_active":"2024-09-13T09:20:11.816376+0000","last_peered":"2024-09-13T09:20:11.816376+0000","last_clean":"2024-09-13T09:20:11.816376+0000","last_became_active":"2024-09-13T09:19:14.748293+0000","last_became_peered":"2024-09-13T09:19:14.748293+0000","last_unstale":"2024-09-13T09:20:11.816376+0000","last_undegraded":"2024-09-13T09:20:11.816376+0000","last_fullsized":"2024-09-13T09:20:11.816376+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:05.560046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016656990000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2578,"num_read_kb":17393,"num_write":357,"num_write_kb":10507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":7471802,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":145,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.791561+0000","last_change":"2024-09-13T09:19:17.770210+0000","last_active":"2024-09-13T09:20:12.791561+0000","last_peered":"2024-09-13T09:20:12.791561+0000","last_clean":"2024-09-13T09:20:12.791561+0000","last_became_active":"2024-09-13T09:19:17.769570+0000","last_became_peered":"2024-09-13T09:19:17.769570+0000","last_unstale":"2024-09-13T09:20:12.791561+0000","last_undegraded":"2024-09-13T09:20:12.791561+0000","last_fullsized":"2024-09-13T09:20:12.791561+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:37.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"147'561","reported_seq":3012,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845458+0000","last_change":"2024-09-13T09:19:51.712759+0000","last_active":"2024-09-13T09:20:10.845458+0000","last_peered":"2024-09-13T09:20:10.845458+0000","last_clean":"2024-09-13T09:20:10.845458+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:20:10.845458+0000","last_undegraded":"2024-09-13T09:20:10.845458+0000","last_fullsized":"2024-09-13T09:20:10.845458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:51.225828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015869739999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4402,"num_read_kb":8559,"num_write":727,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338299+0000","last_change":"2024-09-13T09:17:40.794662+0000","last_active":"2024-09-13T09:20:12.338299+0000","last_peered":"2024-09-13T09:20:12.338299+0000","last_clean":"2024-09-13T09:20:12.338299+0000","last_became_active":"2024-09-13T09:17:40.793882+0000","last_became_peered":"2024-09-13T09:17:40.793882+0000","last_unstale":"2024-09-13T09:20:12.338299+0000","last_undegraded":"2024-09-13T09:20:12.338299+0000","last_fullsized":"2024-09-13T09:20:12.338299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:22.922753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"148'377","reported_seq":1622,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845257+0000","last_change":"2024-09-13T09:19:51.709972+0000","last_active":"2024-09-13T09:20:10.845257+0000","last_peered":"2024-09-13T09:20:10.845257+0000","last_clean":"2024-09-13T09:20:10.845257+0000","last_became_active":"2024-09-13T09:18:03.973314+0000","last_became_peered":"2024-09-13T09:18:03.973314+0000","last_unstale":"2024-09-13T09:20:10.845257+0000","last_undegraded":"2024-09-13T09:20:10.845257+0000","last_fullsized":"2024-09-13T09:20:10.845257+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.256457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012376119999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1399,"num_read_kb":26203,"num_write":287,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":167476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844907+0000","last_change":"2024-09-13T09:18:03.968679+0000","last_active":"2024-09-13T09:20:10.844907+0000","last_peered":"2024-09-13T09:20:10.844907+0000","last_clean":"2024-09-13T09:20:10.844907+0000","last_became_active":"2024-09-13T09:18:03.967133+0000","last_became_peered":"2024-09-13T09:18:03.967133+0000","last_unstale":"2024-09-13T09:20:10.844907+0000","last_undegraded":"2024-09-13T09:20:10.844907+0000","last_fullsized":"2024-09-13T09:20:10.844907+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:06.643934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"146'344","reported_seq":2268,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844102+0000","last_change":"2024-09-13T09:19:51.713872+0000","last_active":"2024-09-13T09:20:10.844102+0000","last_peered":"2024-09-13T09:20:10.844102+0000","last_clean":"2024-09-13T09:20:10.844102+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:20:10.844102+0000","last_undegraded":"2024-09-13T09:20:10.844102+0000","last_fullsized":"2024-09-13T09:20:10.844102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:00.161632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00109892,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2142,"num_read_kb":9020,"num_write":356,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"149'655","reported_seq":4751,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335893+0000","last_change":"2024-09-13T09:19:51.709444+0000","last_active":"2024-09-13T09:20:12.335893+0000","last_peered":"2024-09-13T09:20:12.335893+0000","last_clean":"2024-09-13T09:20:12.335893+0000","last_became_active":"2024-09-13T09:17:40.800027+0000","last_became_peered":"2024-09-13T09:17:40.800027+0000","last_unstale":"2024-09-13T09:20:12.335893+0000","last_undegraded":"2024-09-13T09:20:12.335893+0000","last_fullsized":"2024-09-13T09:20:12.335893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:10.182783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072429099999999995,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":5951,"num_read_kb":14391,"num_write":711,"num_write_kb":6974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":29248,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":199,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.843866+0000","last_change":"2024-09-13T09:18:03.976890+0000","last_active":"2024-09-13T09:20:10.843866+0000","last_peered":"2024-09-13T09:20:10.843866+0000","last_clean":"2024-09-13T09:20:10.843866+0000","last_became_active":"2024-09-13T09:18:03.970835+0000","last_became_peered":"2024-09-13T09:18:03.970835+0000","last_unstale":"2024-09-13T09:20:10.843866+0000","last_undegraded":"2024-09-13T09:20:10.843866+0000","last_fullsized":"2024-09-13T09:20:10.843866+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:05.189083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"147'561","reported_seq":2680,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845372+0000","last_change":"2024-09-13T09:19:51.709196+0000","last_active":"2024-09-13T09:20:10.845372+0000","last_peered":"2024-09-13T09:20:10.845372+0000","last_clean":"2024-09-13T09:20:10.845372+0000","last_became_active":"2024-09-13T09:18:03.970479+0000","last_became_peered":"2024-09-13T09:18:03.970479+0000","last_unstale":"2024-09-13T09:20:10.845372+0000","last_undegraded":"2024-09-13T09:20:10.845372+0000","last_fullsized":"2024-09-13T09:20:10.845372+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:06.229840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001031719,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5538,"num_read_kb":12570,"num_write":635,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1031390,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:09.833063+0000","last_change":"2024-09-13T09:20:02.757363+0000","last_active":"2024-09-13T09:20:09.833063+0000","last_peered":"2024-09-13T09:20:09.833063+0000","last_clean":"2024-09-13T09:20:09.833063+0000","last_became_active":"2024-09-13T09:20:02.756499+0000","last_became_peered":"2024-09-13T09:20:02.756499+0000","last_unstale":"2024-09-13T09:20:09.833063+0000","last_undegraded":"2024-09-13T09:20:09.833063+0000","last_fullsized":"2024-09-13T09:20:09.833063+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:36.829717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"147'483","reported_seq":3446,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816428+0000","last_change":"2024-09-13T09:19:51.724930+0000","last_active":"2024-09-13T09:20:11.816428+0000","last_peered":"2024-09-13T09:20:11.816428+0000","last_clean":"2024-09-13T09:20:11.816428+0000","last_became_active":"2024-09-13T09:17:40.793065+0000","last_became_peered":"2024-09-13T09:17:40.793065+0000","last_unstale":"2024-09-13T09:20:11.816428+0000","last_undegraded":"2024-09-13T09:20:11.816428+0000","last_fullsized":"2024-09-13T09:20:11.816428+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:38.586460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00175362,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4025,"num_read_kb":26901,"num_write":443,"num_write_kb":18254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":178,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336693+0000","last_change":"2024-09-13T09:18:03.968508+0000","last_active":"2024-09-13T09:20:12.336693+0000","last_peered":"2024-09-13T09:20:12.336693+0000","last_clean":"2024-09-13T09:20:12.336693+0000","last_became_active":"2024-09-13T09:18:03.968092+0000","last_became_peered":"2024-09-13T09:18:03.968092+0000","last_unstale":"2024-09-13T09:20:12.336693+0000","last_undegraded":"2024-09-13T09:20:12.336693+0000","last_fullsized":"2024-09-13T09:20:12.336693+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:39.649827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"148'782","reported_seq":2787,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844329+0000","last_change":"2024-09-13T09:19:51.712266+0000","last_active":"2024-09-13T09:20:10.844329+0000","last_peered":"2024-09-13T09:20:10.844329+0000","last_clean":"2024-09-13T09:20:10.844329+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:20:10.844329+0000","last_undegraded":"2024-09-13T09:20:10.844329+0000","last_fullsized":"2024-09-13T09:20:10.844329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:44.094850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012273519999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4598,"num_read_kb":13738,"num_write":1068,"num_write_kb":11812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":162,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.843866+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:20:10.843866+0000","last_peered":"2024-09-13T09:20:10.843866+0000","last_clean":"2024-09-13T09:20:10.843866+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:20:10.843866+0000","last_undegraded":"2024-09-13T09:20:10.843866+0000","last_fullsized":"2024-09-13T09:20:10.843866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":352,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336826+0000","last_change":"2024-09-13T09:17:43.803525+0000","last_active":"2024-09-13T09:20:12.336826+0000","last_peered":"2024-09-13T09:20:12.336826+0000","last_clean":"2024-09-13T09:20:12.336826+0000","last_became_active":"2024-09-13T09:17:43.802603+0000","last_became_peered":"2024-09-13T09:17:43.802603+0000","last_unstale":"2024-09-13T09:20:12.336826+0000","last_undegraded":"2024-09-13T09:20:12.336826+0000","last_fullsized":"2024-09-13T09:20:12.336826+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:07.561959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"147'650","reported_seq":2724,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844330+0000","last_change":"2024-09-13T09:19:51.713229+0000","last_active":"2024-09-13T09:20:10.844330+0000","last_peered":"2024-09-13T09:20:10.844330+0000","last_clean":"2024-09-13T09:20:10.844330+0000","last_became_active":"2024-09-13T09:18:03.974569+0000","last_became_peered":"2024-09-13T09:18:03.974569+0000","last_unstale":"2024-09-13T09:20:10.844330+0000","last_undegraded":"2024-09-13T09:20:10.844330+0000","last_fullsized":"2024-09-13T09:20:10.844330+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:43.479751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036429610000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4126,"num_read_kb":16863,"num_write":738,"num_write_kb":13412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":741496,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"148'539","reported_seq":2812,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335714+0000","last_change":"2024-09-13T09:19:51.709415+0000","last_active":"2024-09-13T09:20:12.335714+0000","last_peered":"2024-09-13T09:20:12.335714+0000","last_clean":"2024-09-13T09:20:12.335714+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:20:12.335714+0000","last_undegraded":"2024-09-13T09:20:12.335714+0000","last_fullsized":"2024-09-13T09:20:12.335714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:06.071033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070054600000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":11592,"num_write":623,"num_write_kb":5754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"148'505","reported_seq":3878,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902417+0000","last_change":"2024-09-13T09:19:51.826968+0000","last_active":"2024-09-13T09:20:08.902417+0000","last_peered":"2024-09-13T09:20:08.902417+0000","last_clean":"2024-09-13T09:20:08.902417+0000","last_became_active":"2024-09-13T09:19:34.891064+0000","last_became_peered":"2024-09-13T09:19:34.891064+0000","last_unstale":"2024-09-13T09:20:08.902417+0000","last_undegraded":"2024-09-13T09:20:08.902417+0000","last_fullsized":"2024-09-13T09:20:08.902417+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:11.410988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078961199999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":3950,"num_read_kb":15857,"num_write":467,"num_write_kb":7750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":4249683,"num_keys_recovered":143,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"149'429","reported_seq":1836,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844608+0000","last_change":"2024-09-13T09:19:51.712307+0000","last_active":"2024-09-13T09:20:10.844608+0000","last_peered":"2024-09-13T09:20:10.844608+0000","last_clean":"2024-09-13T09:20:10.844608+0000","last_became_active":"2024-09-13T09:17:40.801195+0000","last_became_peered":"2024-09-13T09:17:40.801195+0000","last_unstale":"2024-09-13T09:20:10.844608+0000","last_undegraded":"2024-09-13T09:20:10.844608+0000","last_fullsized":"2024-09-13T09:20:10.844608+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:00:07.773428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195911,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2626,"num_read_kb":8389,"num_write":522,"num_write_kb":11747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"149'683","reported_seq":1687,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337358+0000","last_change":"2024-09-13T09:19:51.722579+0000","last_active":"2024-09-13T09:20:12.337358+0000","last_peered":"2024-09-13T09:20:12.337358+0000","last_clean":"2024-09-13T09:20:12.337358+0000","last_became_active":"2024-09-13T09:19:49.736505+0000","last_became_peered":"2024-09-13T09:19:49.736505+0000","last_unstale":"2024-09-13T09:20:12.337358+0000","last_undegraded":"2024-09-13T09:20:12.337358+0000","last_fullsized":"2024-09-13T09:20:12.337358+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:30.617730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00218943,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2689,"num_read_kb":8298,"num_write":946,"num_write_kb":9158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":43,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"147'693","reported_seq":3232,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338076+0000","last_change":"2024-09-13T09:19:51.724802+0000","last_active":"2024-09-13T09:20:12.338076+0000","last_peered":"2024-09-13T09:20:12.338076+0000","last_clean":"2024-09-13T09:20:12.338076+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:20:12.338076+0000","last_undegraded":"2024-09-13T09:20:12.338076+0000","last_fullsized":"2024-09-13T09:20:12.338076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:25.411265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035729350000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4417,"num_read_kb":31654,"num_write":734,"num_write_kb":30325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"146'383","reported_seq":1377,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338412+0000","last_change":"2024-09-13T09:19:51.721061+0000","last_active":"2024-09-13T09:20:12.338412+0000","last_peered":"2024-09-13T09:20:12.338412+0000","last_clean":"2024-09-13T09:20:12.338412+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:20:12.338412+0000","last_undegraded":"2024-09-13T09:20:12.338412+0000","last_fullsized":"2024-09-13T09:20:12.338412+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:31.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068946100000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1301,"num_read_kb":10607,"num_write":406,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"148'980","reported_seq":4053,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338100+0000","last_change":"2024-09-13T09:19:51.722971+0000","last_active":"2024-09-13T09:20:12.338100+0000","last_peered":"2024-09-13T09:20:12.338100+0000","last_clean":"2024-09-13T09:20:12.338100+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:20:12.338100+0000","last_undegraded":"2024-09-13T09:20:12.338100+0000","last_fullsized":"2024-09-13T09:20:12.338100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:56.423075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024735249999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7129,"num_read_kb":22784,"num_write":1446,"num_write_kb":10311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"146'402","reported_seq":2308,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902107+0000","last_change":"2024-09-13T09:19:51.826873+0000","last_active":"2024-09-13T09:20:08.902107+0000","last_peered":"2024-09-13T09:20:08.902107+0000","last_clean":"2024-09-13T09:20:08.902107+0000","last_became_active":"2024-09-13T09:19:20.803495+0000","last_became_peered":"2024-09-13T09:19:20.803495+0000","last_unstale":"2024-09-13T09:20:08.902107+0000","last_undegraded":"2024-09-13T09:20:08.902107+0000","last_fullsized":"2024-09-13T09:20:08.902107+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:18.288761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072899800000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2117,"num_read_kb":10151,"num_write":496,"num_write_kb":12845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5102507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"148'549","reported_seq":3010,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336735+0000","last_change":"2024-09-13T09:19:51.722468+0000","last_active":"2024-09-13T09:20:12.336735+0000","last_peered":"2024-09-13T09:20:12.336735+0000","last_clean":"2024-09-13T09:20:12.336735+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:20:12.336735+0000","last_undegraded":"2024-09-13T09:20:12.336735+0000","last_fullsized":"2024-09-13T09:20:12.336735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:47.572839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020210359999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":14560,"num_write":713,"num_write_kb":15391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"146'438","reported_seq":4935,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816485+0000","last_change":"2024-09-13T09:19:51.724223+0000","last_active":"2024-09-13T09:20:11.816485+0000","last_peered":"2024-09-13T09:20:11.816485+0000","last_clean":"2024-09-13T09:20:11.816485+0000","last_became_active":"2024-09-13T09:17:40.789499+0000","last_became_peered":"2024-09-13T09:17:40.789499+0000","last_unstale":"2024-09-13T09:20:11.816485+0000","last_undegraded":"2024-09-13T09:20:11.816485+0000","last_fullsized":"2024-09-13T09:20:11.816485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:13.395268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010644070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5803,"num_read_kb":44996,"num_write":398,"num_write_kb":17054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"147'421","reported_seq":3620,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337093+0000","last_change":"2024-09-13T09:19:51.723691+0000","last_active":"2024-09-13T09:20:12.337093+0000","last_peered":"2024-09-13T09:20:12.337093+0000","last_clean":"2024-09-13T09:20:12.337093+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:20:12.337093+0000","last_undegraded":"2024-09-13T09:20:12.337093+0000","last_fullsized":"2024-09-13T09:20:12.337093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:07.893848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003077476,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4122,"num_read_kb":18425,"num_write":408,"num_write_kb":9869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":152,"num_read":126915,"num_read_kb":555724,"num_write":19133,"num_write_kb":379149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1469,"num_bytes_recovered":28693609,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16684,"ondisk_log_size":16684,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":30095,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59380,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312460,"statfs":{"total":96636764160,"available":96575959040,"internally_reserved":0,"allocated":2068480,"data_stored":1548192,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":0.94799999999999995,"15min":0.88900000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":11.683,"5min":11.683,"15min":11.683},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.03,"15min":0.98199999999999998},"min":{"1min":0.48099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":11.843999999999999,"5min":11.843999999999999,"15min":11.843999999999999},"last":0.83799999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.86299999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.222,"5min":1.286,"15min":1.286},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.95199999999999996,"15min":0.98399999999999999},"min":{"1min":0.51200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.089,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":1.119}]},{"osd":2,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80000000000000004,"15min":0.80300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.153},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.94899999999999995,"15min":0.95899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.24,"5min":1.385,"15min":1.385},"last":1.0009999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.038,"15min":0.97399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":12.012,"5min":12.012,"15min":12.012},"last":0.873},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.135,"15min":1.075},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":11.776999999999999,"5min":11.776999999999999,"15min":11.776999999999999},"last":1.28}]},{"osd":4,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":1.028,"15min":1.0609999999999999},"min":{"1min":0.44400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2969999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.4590000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.0649999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.073,"15min":1.071},"min":{"1min":0.61199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.3540000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.121,"15min":1.145},"min":{"1min":0.51900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":3.125,"15min":3.125},"last":1.2190000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.1950000000000001,"15min":1.2330000000000001},"min":{"1min":0.59199999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.667,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.109,"15min":1.119},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.498,"5min":1.996,"15min":1.996},"last":1.3100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41272,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330568,"statfs":{"total":96636764160,"available":96594501632,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.438,"5min":0.314,"15min":0.314},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.877},"min":{"1min":0.40899999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3580000000000001,"5min":2.222,"15min":2.222},"last":0.59699999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.83399999999999996,"15min":0.81200000000000006},"min":{"1min":0.50800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.98199999999999998,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.607,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.65700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4319999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.98499999999999999,"15min":1.0029999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.613,"5min":2.508,"15min":2.508},"last":0.70999999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.66900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.55,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.878},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.2450000000000001,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.83799999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.95199999999999996,"15min":0.97399999999999998},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2669999999999999,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.008,"15min":1},"min":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5800000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.92900000000000005}]},{"osd":5,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.92300000000000004,"15min":0.90000000000000002},"min":{"1min":0.58999999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":1.02},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.57599999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.8120000000000001,"5min":2.286,"15min":2.286},"last":1.0720000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.093,"15min":1.087},"min":{"1min":0.621,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.611,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.079,"15min":1.0489999999999999},"min":{"1min":0.69899999999999995,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.1719999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55236,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316604,"statfs":{"total":96636764160,"available":96580202496,"internally_reserved":0,"allocated":2015232,"data_stored":957843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":6.3970000000000002,"15min":7.3419999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1739999999999999,"5min":132.32900000000001,"15min":132.32900000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.236,"5min":1.335,"15min":1.335},"last":1.0589999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.86499999999999999,"15min":0.88500000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96399999999999997,"5min":1.946,"15min":1.946},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.998,"15min":1.0209999999999999},"min":{"1min":0.56899999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1579999999999999,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.71199999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.85099999999999998,"15min":0.84699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1220000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.88800000000000001,"15min":0.89700000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1719999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.98199999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.93799999999999994,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2849999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.028},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0009999999999999,"15min":1.006},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4419999999999999,"5min":1.476,"15min":1.476},"last":0.77500000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86899999999999999,"15min":0.86499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3140000000000001,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":6.8070000000000004,"15min":7.7770000000000001},"min":{"1min":0.47699999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.544,"5min":132.23599999999999,"15min":132.23599999999999},"last":1.135}]},{"osd":6,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.016,"15min":1.0329999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3160000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.079}]},{"osd":7,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.075,"15min":1.075},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.452,"5min":1.655,"15min":1.655},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1279999999999999,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6379999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":0.94999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51136,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320704,"statfs":{"total":96636764160,"available":96584400896,"internally_reserved":0,"allocated":2015232,"data_stored":957843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3560000000000001,"5min":1.357,"15min":1.357},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.94599999999999995,"15min":0.94799999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.331}]},{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91400000000000003,"15min":0.92600000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.81100000000000005,"15min":0.81999999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.877},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.006,"5min":0.88,"15min":0.84299999999999997},"min":{"1min":0.56999999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.111}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86899999999999999,"15min":0.85399999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3300000000000001,"5min":1.575,"15min":1.575},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.069,"5min":1.0249999999999999,"15min":0.97799999999999998},"min":{"1min":0.628,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0899999999999999,"5min":4.7389999999999999,"15min":4.7389999999999999},"last":1.0580000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.45400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.3720000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0329999999999999},"min":{"1min":0.53500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.077,"15min":1.0629999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.4119999999999999,"5min":2.4119999999999999,"15min":2.4119999999999999},"last":1.5720000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.034,"15min":1.026},"min":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5609999999999999,"5min":1.597,"15min":1.597},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.125,"15min":1.131},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.2769999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77372,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94294468,"statfs":{"total":96636764160,"available":96557535232,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.313,"5min":0.215,"15min":0.215},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.62},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.96399999999999997,"5min":1.625,"15min":1.625},"last":0.754}]},{"osd":1,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.28799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.80600000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.36899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.425,"5min":1.544,"15min":1.544},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.755,"15min":0.76200000000000001},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.98199999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89200000000000002,"15min":0.90600000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.25,"5min":1.669,"15min":1.669},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.93300000000000005,"15min":0.94199999999999995},"min":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.296,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.872}]},{"osd":5,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.97199999999999998},"min":{"1min":0.59999999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.5580000000000001,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.92400000000000004,"15min":0.93200000000000005},"min":{"1min":0.63,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3380000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.90400000000000003}]},{"osd":6,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.628,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.47,"5min":1.748,"15min":1.748},"last":1.038},{"interface":"front","average":{"1min":1.024,"5min":1.0860000000000001,"15min":1.0960000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5620000000000001,"5min":2.484,"15min":2.484},"last":1.1950000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.998,"15min":1.006},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.385,"5min":1.806,"15min":1.806},"last":1.01},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.665,"15min":1.772},"min":{"1min":0.61299999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.3879999999999999,"5min":15.4,"15min":15.4},"last":1.0920000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41272,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330568,"statfs":{"total":96636764160,"available":96594501632,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.621},"min":{"1min":0.247,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1899999999999999,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.62},{"interface":"front","average":{"1min":0.749,"5min":0.72499999999999998,"15min":0.70499999999999996},"min":{"1min":0.28399999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.671,"5min":2.8490000000000002,"15min":2.8490000000000002},"last":0.93400000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.79000000000000004,"15min":0.79300000000000004},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.337,"5min":1.907,"15min":1.907},"last":1.137}]},{"osd":3,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.77600000000000002,"15min":0.74399999999999999},"min":{"1min":0.48999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.256,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.83899999999999997,"15min":0.871},"min":{"1min":0.34000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.47,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.97399999999999998,"15min":0.96999999999999997},"min":{"1min":0.76200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4470000000000001,"5min":1.782,"15min":1.782},"last":1.27},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.83499999999999996,"15min":0.81699999999999995},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.97999999999999998,"15min":0.96799999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0329999999999999,"15min":1.0289999999999999},"min":{"1min":0.57199999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5580000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.3180000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.55600000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.012,"15min":1.0049999999999999},"min":{"1min":0.623,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.923,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.88100000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.121,"15min":1.133},"min":{"1min":0.68899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.573,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.226},{"interface":"front","average":{"1min":1.071,"5min":1.0049999999999999,"15min":1.0029999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5620000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.84799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":2019328,"data_stored":957862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.70599999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.429,"5min":1.484,"15min":1.484},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65500000000000003,"15min":0.67100000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.091,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.35299999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.212,"5min":1.264,"15min":1.264},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76000000000000001,"15min":0.76700000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.92900000000000005,"15min":0.95899999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3029999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.98299999999999998,"15min":1.0129999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.22,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.0760000000000001,"15min":1.04},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":8.3460000000000001,"5min":8.3460000000000001,"15min":8.3460000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.998,"15min":1.022},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2769999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.024,"15min":1.0489999999999999},"min":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.339,"5min":1.647,"15min":1.647},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.012,"15min":1.0309999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.389,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.03,"15min":1.052},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.345,"5min":1.629,"15min":1.629},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.095,"15min":1.119},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.411,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.90800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":2052096,"data_stored":1548192,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.316,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.325,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.625,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.006,"5min":1.284,"15min":1.284},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92600000000000005,"5min":1.5,"15min":1.5},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.39000000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.129,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.72499999999999998,"15min":0.69899999999999995},"min":{"1min":0.42699999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.452,"5min":2.242,"15min":2.242},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.78900000000000003},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2250000000000001,"5min":1.528,"15min":1.528},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.89200000000000002,"15min":0.877},"min":{"1min":0.53700000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0309999999999999,"15min":0.96599999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.484,"5min":9.5649999999999995,"15min":9.5649999999999995},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0469999999999999,"15min":0.95799999999999996},"min":{"1min":0.67900000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.375,"5min":9.6120000000000001,"15min":9.6120000000000001},"last":0.75},{"interface":"front","average":{"1min":1.036,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.419,"5min":1.476,"15min":1.476},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0620000000000001,"15min":1.085},"min":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4279999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.996,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.482,"5min":1.516,"15min":1.516},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0229999999999999,"15min":1.0309999999999999},"min":{"1min":0.58799999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.5669999999999999,"5min":1.617,"15min":1.617},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.995,"5min":1.1379999999999999,"15min":1.0820000000000001},"min":{"1min":0.67300000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.546,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":0.78600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:20:16.296 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:20:16.607 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:20:16.607 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:20:16.626 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-13T09:20:16.106477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":152,"num_read":127031,"num_read_kb":555824,"num_write":19294,"num_write_kb":381567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1476,"num_bytes_recovered":29874402,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16804,"ondisk_log_size":16804,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":391880,"kb_used_data":15824,"kb_used_omap":12,"kb_used_meta":375923,"kb_avail":754582840,"statfs":{"total":773094113280,"available":772692828160,"internally_reserved":0,"allocated":16203776,"data_stored":8843404,"data_compressed":19717,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12744,"internal_metadata":384945720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.276048"},"pg_stats":[{"pgid":"3.1f","version":"149'406","reported_seq":1883,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816856+0000","last_change":"2024-09-13T09:19:51.725659+0000","last_active":"2024-09-13T09:20:11.816856+0000","last_peered":"2024-09-13T09:20:11.816856+0000","last_clean":"2024-09-13T09:20:11.816856+0000","last_became_active":"2024-09-13T09:17:40.791187+0000","last_became_peered":"2024-09-13T09:17:40.791187+0000","last_unstale":"2024-09-13T09:20:11.816856+0000","last_undegraded":"2024-09-13T09:20:11.816856+0000","last_fullsized":"2024-09-13T09:20:11.816856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:13.280600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001951139,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2539,"num_read_kb":13964,"num_write":479,"num_write_kb":12323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"146'640","reported_seq":3283,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844414+0000","last_change":"2024-09-13T09:19:51.712843+0000","last_active":"2024-09-13T09:20:10.844414+0000","last_peered":"2024-09-13T09:20:10.844414+0000","last_clean":"2024-09-13T09:20:10.844414+0000","last_became_active":"2024-09-13T09:19:20.802379+0000","last_became_peered":"2024-09-13T09:19:20.802379+0000","last_unstale":"2024-09-13T09:20:10.844414+0000","last_undegraded":"2024-09-13T09:20:10.844414+0000","last_fullsized":"2024-09-13T09:20:10.844414+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:20.952086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001400552,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3630,"num_read_kb":10038,"num_write":856,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":8249616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"146'488","reported_seq":3762,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844008+0000","last_change":"2024-09-13T09:19:51.714853+0000","last_active":"2024-09-13T09:20:10.844008+0000","last_peered":"2024-09-13T09:20:10.844008+0000","last_clean":"2024-09-13T09:20:10.844008+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:20:10.844008+0000","last_undegraded":"2024-09-13T09:20:10.844008+0000","last_fullsized":"2024-09-13T09:20:10.844008+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:20.509991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021011329999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5768,"num_read_kb":20052,"num_write":439,"num_write_kb":19543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"149'793","reported_seq":4565,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.843864+0000","last_change":"2024-09-13T09:19:51.715407+0000","last_active":"2024-09-13T09:20:10.843864+0000","last_peered":"2024-09-13T09:20:10.843864+0000","last_clean":"2024-09-13T09:20:10.843864+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:20:10.843864+0000","last_undegraded":"2024-09-13T09:20:10.843864+0000","last_fullsized":"2024-09-13T09:20:10.843864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:05.002231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025684319999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":7779,"num_read_kb":18614,"num_write":881,"num_write_kb":10036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"148'531","reported_seq":4431,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.861804+0000","last_change":"2024-09-13T09:19:51.725489+0000","last_active":"2024-09-13T09:20:11.861804+0000","last_peered":"2024-09-13T09:20:11.861804+0000","last_clean":"2024-09-13T09:20:11.861804+0000","last_became_active":"2024-09-13T09:17:40.799618+0000","last_became_peered":"2024-09-13T09:17:40.799618+0000","last_unstale":"2024-09-13T09:20:11.861804+0000","last_undegraded":"2024-09-13T09:20:11.861804+0000","last_fullsized":"2024-09-13T09:20:11.861804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:00.757881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001823089,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5766,"num_read_kb":19616,"num_write":659,"num_write_kb":9141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":65341,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"146'344","reported_seq":3288,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335538+0000","last_change":"2024-09-13T09:19:51.709691+0000","last_active":"2024-09-13T09:20:12.335538+0000","last_peered":"2024-09-13T09:20:12.335538+0000","last_clean":"2024-09-13T09:20:12.335538+0000","last_became_active":"2024-09-13T09:18:03.973063+0000","last_became_peered":"2024-09-13T09:18:03.973063+0000","last_unstale":"2024-09-13T09:20:12.335538+0000","last_undegraded":"2024-09-13T09:20:12.335538+0000","last_fullsized":"2024-09-13T09:20:12.335538+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:29.911994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082503800000000005,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3197,"num_read_kb":8961,"num_write":344,"num_write_kb":6710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":889520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'715","reported_seq":4250,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844995+0000","last_change":"2024-09-13T09:19:51.709213+0000","last_active":"2024-09-13T09:20:10.844995+0000","last_peered":"2024-09-13T09:20:10.844995+0000","last_clean":"2024-09-13T09:20:10.844995+0000","last_became_active":"2024-09-13T09:18:06.975016+0000","last_became_peered":"2024-09-13T09:18:06.975016+0000","last_unstale":"2024-09-13T09:20:10.844995+0000","last_undegraded":"2024-09-13T09:20:10.844995+0000","last_fullsized":"2024-09-13T09:20:10.844995+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:30.489567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010502549999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5493,"num_read_kb":28565,"num_write":1011,"num_write_kb":11903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":439303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"149'217","reported_seq":2561,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844055+0000","last_change":"2024-09-13T09:19:51.710337+0000","last_active":"2024-09-13T09:20:10.844055+0000","last_peered":"2024-09-13T09:20:10.844055+0000","last_clean":"2024-09-13T09:20:10.844055+0000","last_became_active":"2024-09-13T09:17:40.797607+0000","last_became_peered":"2024-09-13T09:17:40.797607+0000","last_unstale":"2024-09-13T09:20:10.844055+0000","last_undegraded":"2024-09-13T09:20:10.844055+0000","last_fullsized":"2024-09-13T09:20:10.844055+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:09.025976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011389530000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2065,"num_read_kb":17454,"num_write":181,"num_write_kb":10975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":42856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"148'442","reported_seq":2301,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816853+0000","last_change":"2024-09-13T09:19:51.725896+0000","last_active":"2024-09-13T09:20:11.816853+0000","last_peered":"2024-09-13T09:20:11.816853+0000","last_clean":"2024-09-13T09:20:11.816853+0000","last_became_active":"2024-09-13T09:17:43.809533+0000","last_became_peered":"2024-09-13T09:17:43.809533+0000","last_unstale":"2024-09-13T09:20:11.816853+0000","last_undegraded":"2024-09-13T09:20:11.816853+0000","last_fullsized":"2024-09-13T09:20:11.816853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:20.389195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018671040000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3080,"num_read_kb":24506,"num_write":379,"num_write_kb":12955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":16384,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"148'272","reported_seq":2284,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844180+0000","last_change":"2024-09-13T09:19:51.714263+0000","last_active":"2024-09-13T09:20:10.844180+0000","last_peered":"2024-09-13T09:20:10.844180+0000","last_clean":"2024-09-13T09:20:10.844180+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:20:10.844180+0000","last_undegraded":"2024-09-13T09:20:10.844180+0000","last_fullsized":"2024-09-13T09:20:10.844180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:29.849557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001510742,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1901,"num_read_kb":15086,"num_write":247,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"147'371","reported_seq":2968,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337807+0000","last_change":"2024-09-13T09:19:51.723974+0000","last_active":"2024-09-13T09:20:12.337807+0000","last_peered":"2024-09-13T09:20:12.337807+0000","last_clean":"2024-09-13T09:20:12.337807+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:20:12.337807+0000","last_undegraded":"2024-09-13T09:20:12.337807+0000","last_fullsized":"2024-09-13T09:20:12.337807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:10.717628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029658509999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3481,"num_read_kb":18288,"num_write":412,"num_write_kb":15933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"142'580","reported_seq":3079,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336038+0000","last_change":"2024-09-13T09:19:51.709951+0000","last_active":"2024-09-13T09:20:12.336038+0000","last_peered":"2024-09-13T09:20:12.336038+0000","last_clean":"2024-09-13T09:20:12.336038+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:20:12.336038+0000","last_undegraded":"2024-09-13T09:20:12.336038+0000","last_fullsized":"2024-09-13T09:20:12.336038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:42.341412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048134,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3471,"num_read_kb":17629,"num_write":764,"num_write_kb":18883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902086+0000","last_change":"2024-09-13T09:19:17.796958+0000","last_active":"2024-09-13T09:20:08.902086+0000","last_peered":"2024-09-13T09:20:08.902086+0000","last_clean":"2024-09-13T09:20:08.902086+0000","last_became_active":"2024-09-13T09:19:17.796177+0000","last_became_peered":"2024-09-13T09:19:17.796177+0000","last_unstale":"2024-09-13T09:20:08.902086+0000","last_undegraded":"2024-09-13T09:20:08.902086+0000","last_fullsized":"2024-09-13T09:20:08.902086+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:59.576027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"149'450","reported_seq":1829,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816376+0000","last_change":"2024-09-13T09:19:51.725161+0000","last_active":"2024-09-13T09:20:11.816376+0000","last_peered":"2024-09-13T09:20:11.816376+0000","last_clean":"2024-09-13T09:20:11.816376+0000","last_became_active":"2024-09-13T09:19:14.748293+0000","last_became_peered":"2024-09-13T09:19:14.748293+0000","last_unstale":"2024-09-13T09:20:11.816376+0000","last_undegraded":"2024-09-13T09:20:11.816376+0000","last_fullsized":"2024-09-13T09:20:11.816376+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:05.560046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016656990000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2578,"num_read_kb":17393,"num_write":357,"num_write_kb":10507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":7471802,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":145,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.791561+0000","last_change":"2024-09-13T09:19:17.770210+0000","last_active":"2024-09-13T09:20:12.791561+0000","last_peered":"2024-09-13T09:20:12.791561+0000","last_clean":"2024-09-13T09:20:12.791561+0000","last_became_active":"2024-09-13T09:19:17.769570+0000","last_became_peered":"2024-09-13T09:19:17.769570+0000","last_unstale":"2024-09-13T09:20:12.791561+0000","last_undegraded":"2024-09-13T09:20:12.791561+0000","last_fullsized":"2024-09-13T09:20:12.791561+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:37.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"147'561","reported_seq":3012,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845458+0000","last_change":"2024-09-13T09:19:51.712759+0000","last_active":"2024-09-13T09:20:10.845458+0000","last_peered":"2024-09-13T09:20:10.845458+0000","last_clean":"2024-09-13T09:20:10.845458+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:20:10.845458+0000","last_undegraded":"2024-09-13T09:20:10.845458+0000","last_fullsized":"2024-09-13T09:20:10.845458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:51.225828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015869739999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4402,"num_read_kb":8559,"num_write":727,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338299+0000","last_change":"2024-09-13T09:17:40.794662+0000","last_active":"2024-09-13T09:20:12.338299+0000","last_peered":"2024-09-13T09:20:12.338299+0000","last_clean":"2024-09-13T09:20:12.338299+0000","last_became_active":"2024-09-13T09:17:40.793882+0000","last_became_peered":"2024-09-13T09:17:40.793882+0000","last_unstale":"2024-09-13T09:20:12.338299+0000","last_undegraded":"2024-09-13T09:20:12.338299+0000","last_fullsized":"2024-09-13T09:20:12.338299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:22.922753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"148'377","reported_seq":1622,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845257+0000","last_change":"2024-09-13T09:19:51.709972+0000","last_active":"2024-09-13T09:20:10.845257+0000","last_peered":"2024-09-13T09:20:10.845257+0000","last_clean":"2024-09-13T09:20:10.845257+0000","last_became_active":"2024-09-13T09:18:03.973314+0000","last_became_peered":"2024-09-13T09:18:03.973314+0000","last_unstale":"2024-09-13T09:20:10.845257+0000","last_undegraded":"2024-09-13T09:20:10.845257+0000","last_fullsized":"2024-09-13T09:20:10.845257+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.256457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012376119999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1399,"num_read_kb":26203,"num_write":287,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":167476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844907+0000","last_change":"2024-09-13T09:18:03.968679+0000","last_active":"2024-09-13T09:20:10.844907+0000","last_peered":"2024-09-13T09:20:10.844907+0000","last_clean":"2024-09-13T09:20:10.844907+0000","last_became_active":"2024-09-13T09:18:03.967133+0000","last_became_peered":"2024-09-13T09:18:03.967133+0000","last_unstale":"2024-09-13T09:20:10.844907+0000","last_undegraded":"2024-09-13T09:20:10.844907+0000","last_fullsized":"2024-09-13T09:20:10.844907+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:06.643934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"146'344","reported_seq":2268,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844102+0000","last_change":"2024-09-13T09:19:51.713872+0000","last_active":"2024-09-13T09:20:10.844102+0000","last_peered":"2024-09-13T09:20:10.844102+0000","last_clean":"2024-09-13T09:20:10.844102+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:20:10.844102+0000","last_undegraded":"2024-09-13T09:20:10.844102+0000","last_fullsized":"2024-09-13T09:20:10.844102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:00.161632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00109892,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2142,"num_read_kb":9020,"num_write":356,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"149'655","reported_seq":4751,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335893+0000","last_change":"2024-09-13T09:19:51.709444+0000","last_active":"2024-09-13T09:20:12.335893+0000","last_peered":"2024-09-13T09:20:12.335893+0000","last_clean":"2024-09-13T09:20:12.335893+0000","last_became_active":"2024-09-13T09:17:40.800027+0000","last_became_peered":"2024-09-13T09:17:40.800027+0000","last_unstale":"2024-09-13T09:20:12.335893+0000","last_undegraded":"2024-09-13T09:20:12.335893+0000","last_fullsized":"2024-09-13T09:20:12.335893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:10.182783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072429099999999995,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":5951,"num_read_kb":14391,"num_write":711,"num_write_kb":6974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":29248,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":199,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.843866+0000","last_change":"2024-09-13T09:18:03.976890+0000","last_active":"2024-09-13T09:20:10.843866+0000","last_peered":"2024-09-13T09:20:10.843866+0000","last_clean":"2024-09-13T09:20:10.843866+0000","last_became_active":"2024-09-13T09:18:03.970835+0000","last_became_peered":"2024-09-13T09:18:03.970835+0000","last_unstale":"2024-09-13T09:20:10.843866+0000","last_undegraded":"2024-09-13T09:20:10.843866+0000","last_fullsized":"2024-09-13T09:20:10.843866+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:05.189083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"147'561","reported_seq":2680,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845372+0000","last_change":"2024-09-13T09:19:51.709196+0000","last_active":"2024-09-13T09:20:10.845372+0000","last_peered":"2024-09-13T09:20:10.845372+0000","last_clean":"2024-09-13T09:20:10.845372+0000","last_became_active":"2024-09-13T09:18:03.970479+0000","last_became_peered":"2024-09-13T09:18:03.970479+0000","last_unstale":"2024-09-13T09:20:10.845372+0000","last_undegraded":"2024-09-13T09:20:10.845372+0000","last_fullsized":"2024-09-13T09:20:10.845372+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:06.229840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001031719,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5538,"num_read_kb":12570,"num_write":635,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1031390,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:09.833063+0000","last_change":"2024-09-13T09:20:02.757363+0000","last_active":"2024-09-13T09:20:09.833063+0000","last_peered":"2024-09-13T09:20:09.833063+0000","last_clean":"2024-09-13T09:20:09.833063+0000","last_became_active":"2024-09-13T09:20:02.756499+0000","last_became_peered":"2024-09-13T09:20:02.756499+0000","last_unstale":"2024-09-13T09:20:09.833063+0000","last_undegraded":"2024-09-13T09:20:09.833063+0000","last_fullsized":"2024-09-13T09:20:09.833063+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:36.829717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"147'483","reported_seq":3446,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816428+0000","last_change":"2024-09-13T09:19:51.724930+0000","last_active":"2024-09-13T09:20:11.816428+0000","last_peered":"2024-09-13T09:20:11.816428+0000","last_clean":"2024-09-13T09:20:11.816428+0000","last_became_active":"2024-09-13T09:17:40.793065+0000","last_became_peered":"2024-09-13T09:17:40.793065+0000","last_unstale":"2024-09-13T09:20:11.816428+0000","last_undegraded":"2024-09-13T09:20:11.816428+0000","last_fullsized":"2024-09-13T09:20:11.816428+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:38.586460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00175362,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4025,"num_read_kb":26901,"num_write":443,"num_write_kb":18254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":178,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336693+0000","last_change":"2024-09-13T09:18:03.968508+0000","last_active":"2024-09-13T09:20:12.336693+0000","last_peered":"2024-09-13T09:20:12.336693+0000","last_clean":"2024-09-13T09:20:12.336693+0000","last_became_active":"2024-09-13T09:18:03.968092+0000","last_became_peered":"2024-09-13T09:18:03.968092+0000","last_unstale":"2024-09-13T09:20:12.336693+0000","last_undegraded":"2024-09-13T09:20:12.336693+0000","last_fullsized":"2024-09-13T09:20:12.336693+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:39.649827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"148'782","reported_seq":2787,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844329+0000","last_change":"2024-09-13T09:19:51.712266+0000","last_active":"2024-09-13T09:20:10.844329+0000","last_peered":"2024-09-13T09:20:10.844329+0000","last_clean":"2024-09-13T09:20:10.844329+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:20:10.844329+0000","last_undegraded":"2024-09-13T09:20:10.844329+0000","last_fullsized":"2024-09-13T09:20:10.844329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:44.094850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012273519999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4598,"num_read_kb":13738,"num_write":1068,"num_write_kb":11812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":162,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.843866+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:20:10.843866+0000","last_peered":"2024-09-13T09:20:10.843866+0000","last_clean":"2024-09-13T09:20:10.843866+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:20:10.843866+0000","last_undegraded":"2024-09-13T09:20:10.843866+0000","last_fullsized":"2024-09-13T09:20:10.843866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":352,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336826+0000","last_change":"2024-09-13T09:17:43.803525+0000","last_active":"2024-09-13T09:20:12.336826+0000","last_peered":"2024-09-13T09:20:12.336826+0000","last_clean":"2024-09-13T09:20:12.336826+0000","last_became_active":"2024-09-13T09:17:43.802603+0000","last_became_peered":"2024-09-13T09:17:43.802603+0000","last_unstale":"2024-09-13T09:20:12.336826+0000","last_undegraded":"2024-09-13T09:20:12.336826+0000","last_fullsized":"2024-09-13T09:20:12.336826+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:07.561959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"147'650","reported_seq":2724,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844330+0000","last_change":"2024-09-13T09:19:51.713229+0000","last_active":"2024-09-13T09:20:10.844330+0000","last_peered":"2024-09-13T09:20:10.844330+0000","last_clean":"2024-09-13T09:20:10.844330+0000","last_became_active":"2024-09-13T09:18:03.974569+0000","last_became_peered":"2024-09-13T09:18:03.974569+0000","last_unstale":"2024-09-13T09:20:10.844330+0000","last_undegraded":"2024-09-13T09:20:10.844330+0000","last_fullsized":"2024-09-13T09:20:10.844330+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:43.479751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036429610000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4126,"num_read_kb":16863,"num_write":738,"num_write_kb":13412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":741496,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"148'539","reported_seq":2812,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335714+0000","last_change":"2024-09-13T09:19:51.709415+0000","last_active":"2024-09-13T09:20:12.335714+0000","last_peered":"2024-09-13T09:20:12.335714+0000","last_clean":"2024-09-13T09:20:12.335714+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:20:12.335714+0000","last_undegraded":"2024-09-13T09:20:12.335714+0000","last_fullsized":"2024-09-13T09:20:12.335714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:06.071033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070054600000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":11592,"num_write":623,"num_write_kb":5754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"148'505","reported_seq":3878,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902417+0000","last_change":"2024-09-13T09:19:51.826968+0000","last_active":"2024-09-13T09:20:08.902417+0000","last_peered":"2024-09-13T09:20:08.902417+0000","last_clean":"2024-09-13T09:20:08.902417+0000","last_became_active":"2024-09-13T09:19:34.891064+0000","last_became_peered":"2024-09-13T09:19:34.891064+0000","last_unstale":"2024-09-13T09:20:08.902417+0000","last_undegraded":"2024-09-13T09:20:08.902417+0000","last_fullsized":"2024-09-13T09:20:08.902417+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:11.410988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078961199999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":3950,"num_read_kb":15857,"num_write":467,"num_write_kb":7750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":4249683,"num_keys_recovered":143,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"149'429","reported_seq":1836,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844608+0000","last_change":"2024-09-13T09:19:51.712307+0000","last_active":"2024-09-13T09:20:10.844608+0000","last_peered":"2024-09-13T09:20:10.844608+0000","last_clean":"2024-09-13T09:20:10.844608+0000","last_became_active":"2024-09-13T09:17:40.801195+0000","last_became_peered":"2024-09-13T09:17:40.801195+0000","last_unstale":"2024-09-13T09:20:10.844608+0000","last_undegraded":"2024-09-13T09:20:10.844608+0000","last_fullsized":"2024-09-13T09:20:10.844608+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:00:07.773428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195911,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2626,"num_read_kb":8389,"num_write":522,"num_write_kb":11747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"149'683","reported_seq":1687,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337358+0000","last_change":"2024-09-13T09:19:51.722579+0000","last_active":"2024-09-13T09:20:12.337358+0000","last_peered":"2024-09-13T09:20:12.337358+0000","last_clean":"2024-09-13T09:20:12.337358+0000","last_became_active":"2024-09-13T09:19:49.736505+0000","last_became_peered":"2024-09-13T09:19:49.736505+0000","last_unstale":"2024-09-13T09:20:12.337358+0000","last_undegraded":"2024-09-13T09:20:12.337358+0000","last_fullsized":"2024-09-13T09:20:12.337358+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:30.617730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00218943,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2689,"num_read_kb":8298,"num_write":946,"num_write_kb":9158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":43,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"147'693","reported_seq":3232,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338076+0000","last_change":"2024-09-13T09:19:51.724802+0000","last_active":"2024-09-13T09:20:12.338076+0000","last_peered":"2024-09-13T09:20:12.338076+0000","last_clean":"2024-09-13T09:20:12.338076+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:20:12.338076+0000","last_undegraded":"2024-09-13T09:20:12.338076+0000","last_fullsized":"2024-09-13T09:20:12.338076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:25.411265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035729350000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4417,"num_read_kb":31654,"num_write":734,"num_write_kb":30325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"146'383","reported_seq":1377,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338412+0000","last_change":"2024-09-13T09:19:51.721061+0000","last_active":"2024-09-13T09:20:12.338412+0000","last_peered":"2024-09-13T09:20:12.338412+0000","last_clean":"2024-09-13T09:20:12.338412+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:20:12.338412+0000","last_undegraded":"2024-09-13T09:20:12.338412+0000","last_fullsized":"2024-09-13T09:20:12.338412+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:31.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068946100000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1301,"num_read_kb":10607,"num_write":406,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"148'980","reported_seq":4053,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338100+0000","last_change":"2024-09-13T09:19:51.722971+0000","last_active":"2024-09-13T09:20:12.338100+0000","last_peered":"2024-09-13T09:20:12.338100+0000","last_clean":"2024-09-13T09:20:12.338100+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:20:12.338100+0000","last_undegraded":"2024-09-13T09:20:12.338100+0000","last_fullsized":"2024-09-13T09:20:12.338100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:56.423075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024735249999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7129,"num_read_kb":22784,"num_write":1446,"num_write_kb":10311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"146'402","reported_seq":2308,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902107+0000","last_change":"2024-09-13T09:19:51.826873+0000","last_active":"2024-09-13T09:20:08.902107+0000","last_peered":"2024-09-13T09:20:08.902107+0000","last_clean":"2024-09-13T09:20:08.902107+0000","last_became_active":"2024-09-13T09:19:20.803495+0000","last_became_peered":"2024-09-13T09:19:20.803495+0000","last_unstale":"2024-09-13T09:20:08.902107+0000","last_undegraded":"2024-09-13T09:20:08.902107+0000","last_fullsized":"2024-09-13T09:20:08.902107+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:18.288761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072899800000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2117,"num_read_kb":10151,"num_write":496,"num_write_kb":12845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5102507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"148'549","reported_seq":3010,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336735+0000","last_change":"2024-09-13T09:19:51.722468+0000","last_active":"2024-09-13T09:20:12.336735+0000","last_peered":"2024-09-13T09:20:12.336735+0000","last_clean":"2024-09-13T09:20:12.336735+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:20:12.336735+0000","last_undegraded":"2024-09-13T09:20:12.336735+0000","last_fullsized":"2024-09-13T09:20:12.336735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:47.572839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020210359999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":14560,"num_write":713,"num_write_kb":15391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"146'438","reported_seq":4935,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816485+0000","last_change":"2024-09-13T09:19:51.724223+0000","last_active":"2024-09-13T09:20:11.816485+0000","last_peered":"2024-09-13T09:20:11.816485+0000","last_clean":"2024-09-13T09:20:11.816485+0000","last_became_active":"2024-09-13T09:17:40.789499+0000","last_became_peered":"2024-09-13T09:17:40.789499+0000","last_unstale":"2024-09-13T09:20:11.816485+0000","last_undegraded":"2024-09-13T09:20:11.816485+0000","last_fullsized":"2024-09-13T09:20:11.816485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:13.395268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010644070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5803,"num_read_kb":44996,"num_write":398,"num_write_kb":17054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"147'421","reported_seq":3620,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337093+0000","last_change":"2024-09-13T09:19:51.723691+0000","last_active":"2024-09-13T09:20:12.337093+0000","last_peered":"2024-09-13T09:20:12.337093+0000","last_clean":"2024-09-13T09:20:12.337093+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:20:12.337093+0000","last_undegraded":"2024-09-13T09:20:12.337093+0000","last_fullsized":"2024-09-13T09:20:12.337093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:07.893848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003077476,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4122,"num_read_kb":18425,"num_write":408,"num_write_kb":9869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":152,"num_read":126915,"num_read_kb":555724,"num_write":19133,"num_write_kb":379149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1469,"num_bytes_recovered":28693609,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16684,"ondisk_log_size":16684,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":30095,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59380,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312460,"statfs":{"total":96636764160,"available":96575959040,"internally_reserved":0,"allocated":2068480,"data_stored":1548192,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":0.94799999999999995,"15min":0.88900000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":11.683,"5min":11.683,"15min":11.683},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.03,"15min":0.98199999999999998},"min":{"1min":0.48099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":11.843999999999999,"5min":11.843999999999999,"15min":11.843999999999999},"last":0.83799999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.86299999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.222,"5min":1.286,"15min":1.286},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.95199999999999996,"15min":0.98399999999999999},"min":{"1min":0.51200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.089,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":1.119}]},{"osd":2,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80000000000000004,"15min":0.80300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.153},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.94899999999999995,"15min":0.95899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.24,"5min":1.385,"15min":1.385},"last":1.0009999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.038,"15min":0.97399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":12.012,"5min":12.012,"15min":12.012},"last":0.873},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.135,"15min":1.075},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":11.776999999999999,"5min":11.776999999999999,"15min":11.776999999999999},"last":1.28}]},{"osd":4,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":1.028,"15min":1.0609999999999999},"min":{"1min":0.44400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2969999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.4590000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.0649999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.073,"15min":1.071},"min":{"1min":0.61199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.3540000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.121,"15min":1.145},"min":{"1min":0.51900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":3.125,"15min":3.125},"last":1.2190000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.1950000000000001,"15min":1.2330000000000001},"min":{"1min":0.59199999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.667,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.109,"15min":1.119},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.498,"5min":1.996,"15min":1.996},"last":1.3100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41272,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330568,"statfs":{"total":96636764160,"available":96594501632,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.438,"5min":0.314,"15min":0.314},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.877},"min":{"1min":0.40899999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3580000000000001,"5min":2.222,"15min":2.222},"last":0.59699999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.83399999999999996,"15min":0.81200000000000006},"min":{"1min":0.50800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.98199999999999998,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.607,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.65700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4319999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.98499999999999999,"15min":1.0029999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.613,"5min":2.508,"15min":2.508},"last":0.70999999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.66900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.55,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.878},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.2450000000000001,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.83799999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.95199999999999996,"15min":0.97399999999999998},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2669999999999999,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.008,"15min":1},"min":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5800000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.92900000000000005}]},{"osd":5,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.92300000000000004,"15min":0.90000000000000002},"min":{"1min":0.58999999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":1.02},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.57599999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.8120000000000001,"5min":2.286,"15min":2.286},"last":1.0720000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.093,"15min":1.087},"min":{"1min":0.621,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.611,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.079,"15min":1.0489999999999999},"min":{"1min":0.69899999999999995,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.1719999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55236,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316604,"statfs":{"total":96636764160,"available":96580202496,"internally_reserved":0,"allocated":2015232,"data_stored":957843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":6.3970000000000002,"15min":7.3419999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1739999999999999,"5min":132.32900000000001,"15min":132.32900000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.236,"5min":1.335,"15min":1.335},"last":1.0589999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.86499999999999999,"15min":0.88500000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96399999999999997,"5min":1.946,"15min":1.946},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.998,"15min":1.0209999999999999},"min":{"1min":0.56899999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1579999999999999,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.71199999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.85099999999999998,"15min":0.84699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1220000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.88800000000000001,"15min":0.89700000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1719999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.98199999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.93799999999999994,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2849999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.028},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0009999999999999,"15min":1.006},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4419999999999999,"5min":1.476,"15min":1.476},"last":0.77500000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86899999999999999,"15min":0.86499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3140000000000001,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":6.8070000000000004,"15min":7.7770000000000001},"min":{"1min":0.47699999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.544,"5min":132.23599999999999,"15min":132.23599999999999},"last":1.135}]},{"osd":6,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.016,"15min":1.0329999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3160000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.079}]},{"osd":7,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.075,"15min":1.075},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.452,"5min":1.655,"15min":1.655},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1279999999999999,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6379999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":0.94999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51136,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320704,"statfs":{"total":96636764160,"available":96584400896,"internally_reserved":0,"allocated":2015232,"data_stored":957843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3560000000000001,"5min":1.357,"15min":1.357},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.94599999999999995,"15min":0.94799999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.331}]},{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91400000000000003,"15min":0.92600000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.81100000000000005,"15min":0.81999999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.877},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.006,"5min":0.88,"15min":0.84299999999999997},"min":{"1min":0.56999999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.111}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86899999999999999,"15min":0.85399999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3300000000000001,"5min":1.575,"15min":1.575},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.069,"5min":1.0249999999999999,"15min":0.97799999999999998},"min":{"1min":0.628,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0899999999999999,"5min":4.7389999999999999,"15min":4.7389999999999999},"last":1.0580000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.45400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.3720000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0329999999999999},"min":{"1min":0.53500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.077,"15min":1.0629999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.4119999999999999,"5min":2.4119999999999999,"15min":2.4119999999999999},"last":1.5720000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.034,"15min":1.026},"min":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5609999999999999,"5min":1.597,"15min":1.597},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.125,"15min":1.131},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.2769999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77372,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94294468,"statfs":{"total":96636764160,"available":96557535232,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.313,"5min":0.215,"15min":0.215},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.62},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.96399999999999997,"5min":1.625,"15min":1.625},"last":0.754}]},{"osd":1,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.28799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.80600000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.36899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.425,"5min":1.544,"15min":1.544},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.755,"15min":0.76200000000000001},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.98199999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89200000000000002,"15min":0.90600000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.25,"5min":1.669,"15min":1.669},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.93300000000000005,"15min":0.94199999999999995},"min":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.296,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.872}]},{"osd":5,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.97199999999999998},"min":{"1min":0.59999999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.5580000000000001,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.92400000000000004,"15min":0.93200000000000005},"min":{"1min":0.63,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3380000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.90400000000000003}]},{"osd":6,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.628,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.47,"5min":1.748,"15min":1.748},"last":1.038},{"interface":"front","average":{"1min":1.024,"5min":1.0860000000000001,"15min":1.0960000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5620000000000001,"5min":2.484,"15min":2.484},"last":1.1950000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.998,"15min":1.006},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.385,"5min":1.806,"15min":1.806},"last":1.01},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.665,"15min":1.772},"min":{"1min":0.61299999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.3879999999999999,"5min":15.4,"15min":15.4},"last":1.0920000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41272,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330568,"statfs":{"total":96636764160,"available":96594501632,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.621},"min":{"1min":0.247,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1899999999999999,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.62},{"interface":"front","average":{"1min":0.749,"5min":0.72499999999999998,"15min":0.70499999999999996},"min":{"1min":0.28399999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.671,"5min":2.8490000000000002,"15min":2.8490000000000002},"last":0.93400000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.79000000000000004,"15min":0.79300000000000004},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.337,"5min":1.907,"15min":1.907},"last":1.137}]},{"osd":3,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.77600000000000002,"15min":0.74399999999999999},"min":{"1min":0.48999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.256,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.83899999999999997,"15min":0.871},"min":{"1min":0.34000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.47,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.97399999999999998,"15min":0.96999999999999997},"min":{"1min":0.76200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4470000000000001,"5min":1.782,"15min":1.782},"last":1.27},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.83499999999999996,"15min":0.81699999999999995},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.97999999999999998,"15min":0.96799999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0329999999999999,"15min":1.0289999999999999},"min":{"1min":0.57199999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5580000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.3180000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.55600000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.012,"15min":1.0049999999999999},"min":{"1min":0.623,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.923,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.88100000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.121,"15min":1.133},"min":{"1min":0.68899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.573,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.226},{"interface":"front","average":{"1min":1.071,"5min":1.0049999999999999,"15min":1.0029999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5620000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.84799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":2019328,"data_stored":957862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.70599999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.429,"5min":1.484,"15min":1.484},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65500000000000003,"15min":0.67100000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.091,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.35299999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.212,"5min":1.264,"15min":1.264},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76000000000000001,"15min":0.76700000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.92900000000000005,"15min":0.95899999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3029999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.98299999999999998,"15min":1.0129999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.22,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.0760000000000001,"15min":1.04},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":8.3460000000000001,"5min":8.3460000000000001,"15min":8.3460000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.998,"15min":1.022},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2769999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.024,"15min":1.0489999999999999},"min":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.339,"5min":1.647,"15min":1.647},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.012,"15min":1.0309999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.389,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.03,"15min":1.052},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.345,"5min":1.629,"15min":1.629},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.095,"15min":1.119},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.411,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.90800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":2052096,"data_stored":1548192,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.316,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.325,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.625,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.006,"5min":1.284,"15min":1.284},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92600000000000005,"5min":1.5,"15min":1.5},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.39000000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.129,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.72499999999999998,"15min":0.69899999999999995},"min":{"1min":0.42699999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.452,"5min":2.242,"15min":2.242},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.78900000000000003},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2250000000000001,"5min":1.528,"15min":1.528},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.89200000000000002,"15min":0.877},"min":{"1min":0.53700000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0309999999999999,"15min":0.96599999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.484,"5min":9.5649999999999995,"15min":9.5649999999999995},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0469999999999999,"15min":0.95799999999999996},"min":{"1min":0.67900000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.375,"5min":9.6120000000000001,"15min":9.6120000000000001},"last":0.75},{"interface":"front","average":{"1min":1.036,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.419,"5min":1.476,"15min":1.476},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0620000000000001,"15min":1.085},"min":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4279999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.996,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.482,"5min":1.516,"15min":1.516},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0229999999999999,"15min":1.0309999999999999},"min":{"1min":0.58799999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.5669999999999999,"5min":1.617,"15min":1.617},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.995,"5min":1.1379999999999999,"15min":1.0820000000000001},"min":{"1min":0.67300000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.546,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":0.78600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:20:16.629 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T09:20:16.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:20:16.936 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:20:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:20:16.955 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-13T09:20:16.106477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":152,"num_read":127031,"num_read_kb":555824,"num_write":19294,"num_write_kb":381567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1476,"num_bytes_recovered":29874402,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16804,"ondisk_log_size":16804,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":391880,"kb_used_data":15824,"kb_used_omap":12,"kb_used_meta":375923,"kb_avail":754582840,"statfs":{"total":773094113280,"available":772692828160,"internally_reserved":0,"allocated":16203776,"data_stored":8843404,"data_compressed":19717,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12744,"internal_metadata":384945720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.276048"},"pg_stats":[{"pgid":"3.1f","version":"149'406","reported_seq":1883,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816856+0000","last_change":"2024-09-13T09:19:51.725659+0000","last_active":"2024-09-13T09:20:11.816856+0000","last_peered":"2024-09-13T09:20:11.816856+0000","last_clean":"2024-09-13T09:20:11.816856+0000","last_became_active":"2024-09-13T09:17:40.791187+0000","last_became_peered":"2024-09-13T09:17:40.791187+0000","last_unstale":"2024-09-13T09:20:11.816856+0000","last_undegraded":"2024-09-13T09:20:11.816856+0000","last_fullsized":"2024-09-13T09:20:11.816856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:13.280600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001951139,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2539,"num_read_kb":13964,"num_write":479,"num_write_kb":12323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"146'640","reported_seq":3283,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844414+0000","last_change":"2024-09-13T09:19:51.712843+0000","last_active":"2024-09-13T09:20:10.844414+0000","last_peered":"2024-09-13T09:20:10.844414+0000","last_clean":"2024-09-13T09:20:10.844414+0000","last_became_active":"2024-09-13T09:19:20.802379+0000","last_became_peered":"2024-09-13T09:19:20.802379+0000","last_unstale":"2024-09-13T09:20:10.844414+0000","last_undegraded":"2024-09-13T09:20:10.844414+0000","last_fullsized":"2024-09-13T09:20:10.844414+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:20.952086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001400552,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3630,"num_read_kb":10038,"num_write":856,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":8249616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"146'488","reported_seq":3762,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844008+0000","last_change":"2024-09-13T09:19:51.714853+0000","last_active":"2024-09-13T09:20:10.844008+0000","last_peered":"2024-09-13T09:20:10.844008+0000","last_clean":"2024-09-13T09:20:10.844008+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:20:10.844008+0000","last_undegraded":"2024-09-13T09:20:10.844008+0000","last_fullsized":"2024-09-13T09:20:10.844008+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:20.509991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021011329999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5768,"num_read_kb":20052,"num_write":439,"num_write_kb":19543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"149'793","reported_seq":4565,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.843864+0000","last_change":"2024-09-13T09:19:51.715407+0000","last_active":"2024-09-13T09:20:10.843864+0000","last_peered":"2024-09-13T09:20:10.843864+0000","last_clean":"2024-09-13T09:20:10.843864+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:20:10.843864+0000","last_undegraded":"2024-09-13T09:20:10.843864+0000","last_fullsized":"2024-09-13T09:20:10.843864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:05.002231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025684319999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":7779,"num_read_kb":18614,"num_write":881,"num_write_kb":10036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"148'531","reported_seq":4431,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.861804+0000","last_change":"2024-09-13T09:19:51.725489+0000","last_active":"2024-09-13T09:20:11.861804+0000","last_peered":"2024-09-13T09:20:11.861804+0000","last_clean":"2024-09-13T09:20:11.861804+0000","last_became_active":"2024-09-13T09:17:40.799618+0000","last_became_peered":"2024-09-13T09:17:40.799618+0000","last_unstale":"2024-09-13T09:20:11.861804+0000","last_undegraded":"2024-09-13T09:20:11.861804+0000","last_fullsized":"2024-09-13T09:20:11.861804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:00.757881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001823089,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5766,"num_read_kb":19616,"num_write":659,"num_write_kb":9141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":65341,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"146'344","reported_seq":3288,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335538+0000","last_change":"2024-09-13T09:19:51.709691+0000","last_active":"2024-09-13T09:20:12.335538+0000","last_peered":"2024-09-13T09:20:12.335538+0000","last_clean":"2024-09-13T09:20:12.335538+0000","last_became_active":"2024-09-13T09:18:03.973063+0000","last_became_peered":"2024-09-13T09:18:03.973063+0000","last_unstale":"2024-09-13T09:20:12.335538+0000","last_undegraded":"2024-09-13T09:20:12.335538+0000","last_fullsized":"2024-09-13T09:20:12.335538+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:29.911994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082503800000000005,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3197,"num_read_kb":8961,"num_write":344,"num_write_kb":6710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":889520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'715","reported_seq":4250,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844995+0000","last_change":"2024-09-13T09:19:51.709213+0000","last_active":"2024-09-13T09:20:10.844995+0000","last_peered":"2024-09-13T09:20:10.844995+0000","last_clean":"2024-09-13T09:20:10.844995+0000","last_became_active":"2024-09-13T09:18:06.975016+0000","last_became_peered":"2024-09-13T09:18:06.975016+0000","last_unstale":"2024-09-13T09:20:10.844995+0000","last_undegraded":"2024-09-13T09:20:10.844995+0000","last_fullsized":"2024-09-13T09:20:10.844995+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:30.489567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010502549999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5493,"num_read_kb":28565,"num_write":1011,"num_write_kb":11903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":439303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"149'217","reported_seq":2561,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844055+0000","last_change":"2024-09-13T09:19:51.710337+0000","last_active":"2024-09-13T09:20:10.844055+0000","last_peered":"2024-09-13T09:20:10.844055+0000","last_clean":"2024-09-13T09:20:10.844055+0000","last_became_active":"2024-09-13T09:17:40.797607+0000","last_became_peered":"2024-09-13T09:17:40.797607+0000","last_unstale":"2024-09-13T09:20:10.844055+0000","last_undegraded":"2024-09-13T09:20:10.844055+0000","last_fullsized":"2024-09-13T09:20:10.844055+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:09.025976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011389530000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2065,"num_read_kb":17454,"num_write":181,"num_write_kb":10975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":42856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"148'442","reported_seq":2301,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816853+0000","last_change":"2024-09-13T09:19:51.725896+0000","last_active":"2024-09-13T09:20:11.816853+0000","last_peered":"2024-09-13T09:20:11.816853+0000","last_clean":"2024-09-13T09:20:11.816853+0000","last_became_active":"2024-09-13T09:17:43.809533+0000","last_became_peered":"2024-09-13T09:17:43.809533+0000","last_unstale":"2024-09-13T09:20:11.816853+0000","last_undegraded":"2024-09-13T09:20:11.816853+0000","last_fullsized":"2024-09-13T09:20:11.816853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:20.389195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018671040000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3080,"num_read_kb":24506,"num_write":379,"num_write_kb":12955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":16384,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"148'272","reported_seq":2284,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844180+0000","last_change":"2024-09-13T09:19:51.714263+0000","last_active":"2024-09-13T09:20:10.844180+0000","last_peered":"2024-09-13T09:20:10.844180+0000","last_clean":"2024-09-13T09:20:10.844180+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:20:10.844180+0000","last_undegraded":"2024-09-13T09:20:10.844180+0000","last_fullsized":"2024-09-13T09:20:10.844180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:29.849557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001510742,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1901,"num_read_kb":15086,"num_write":247,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"147'371","reported_seq":2968,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337807+0000","last_change":"2024-09-13T09:19:51.723974+0000","last_active":"2024-09-13T09:20:12.337807+0000","last_peered":"2024-09-13T09:20:12.337807+0000","last_clean":"2024-09-13T09:20:12.337807+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:20:12.337807+0000","last_undegraded":"2024-09-13T09:20:12.337807+0000","last_fullsized":"2024-09-13T09:20:12.337807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:10.717628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029658509999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3481,"num_read_kb":18288,"num_write":412,"num_write_kb":15933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"142'580","reported_seq":3079,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336038+0000","last_change":"2024-09-13T09:19:51.709951+0000","last_active":"2024-09-13T09:20:12.336038+0000","last_peered":"2024-09-13T09:20:12.336038+0000","last_clean":"2024-09-13T09:20:12.336038+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:20:12.336038+0000","last_undegraded":"2024-09-13T09:20:12.336038+0000","last_fullsized":"2024-09-13T09:20:12.336038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:42.341412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048134,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3471,"num_read_kb":17629,"num_write":764,"num_write_kb":18883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902086+0000","last_change":"2024-09-13T09:19:17.796958+0000","last_active":"2024-09-13T09:20:08.902086+0000","last_peered":"2024-09-13T09:20:08.902086+0000","last_clean":"2024-09-13T09:20:08.902086+0000","last_became_active":"2024-09-13T09:19:17.796177+0000","last_became_peered":"2024-09-13T09:19:17.796177+0000","last_unstale":"2024-09-13T09:20:08.902086+0000","last_undegraded":"2024-09-13T09:20:08.902086+0000","last_fullsized":"2024-09-13T09:20:08.902086+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:59.576027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"149'450","reported_seq":1829,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816376+0000","last_change":"2024-09-13T09:19:51.725161+0000","last_active":"2024-09-13T09:20:11.816376+0000","last_peered":"2024-09-13T09:20:11.816376+0000","last_clean":"2024-09-13T09:20:11.816376+0000","last_became_active":"2024-09-13T09:19:14.748293+0000","last_became_peered":"2024-09-13T09:19:14.748293+0000","last_unstale":"2024-09-13T09:20:11.816376+0000","last_undegraded":"2024-09-13T09:20:11.816376+0000","last_fullsized":"2024-09-13T09:20:11.816376+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:05.560046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016656990000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2578,"num_read_kb":17393,"num_write":357,"num_write_kb":10507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":7471802,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":145,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.791561+0000","last_change":"2024-09-13T09:19:17.770210+0000","last_active":"2024-09-13T09:20:12.791561+0000","last_peered":"2024-09-13T09:20:12.791561+0000","last_clean":"2024-09-13T09:20:12.791561+0000","last_became_active":"2024-09-13T09:19:17.769570+0000","last_became_peered":"2024-09-13T09:19:17.769570+0000","last_unstale":"2024-09-13T09:20:12.791561+0000","last_undegraded":"2024-09-13T09:20:12.791561+0000","last_fullsized":"2024-09-13T09:20:12.791561+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:37.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"147'561","reported_seq":3012,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845458+0000","last_change":"2024-09-13T09:19:51.712759+0000","last_active":"2024-09-13T09:20:10.845458+0000","last_peered":"2024-09-13T09:20:10.845458+0000","last_clean":"2024-09-13T09:20:10.845458+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:20:10.845458+0000","last_undegraded":"2024-09-13T09:20:10.845458+0000","last_fullsized":"2024-09-13T09:20:10.845458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:51.225828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015869739999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4402,"num_read_kb":8559,"num_write":727,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338299+0000","last_change":"2024-09-13T09:17:40.794662+0000","last_active":"2024-09-13T09:20:12.338299+0000","last_peered":"2024-09-13T09:20:12.338299+0000","last_clean":"2024-09-13T09:20:12.338299+0000","last_became_active":"2024-09-13T09:17:40.793882+0000","last_became_peered":"2024-09-13T09:17:40.793882+0000","last_unstale":"2024-09-13T09:20:12.338299+0000","last_undegraded":"2024-09-13T09:20:12.338299+0000","last_fullsized":"2024-09-13T09:20:12.338299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:22.922753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"148'377","reported_seq":1622,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845257+0000","last_change":"2024-09-13T09:19:51.709972+0000","last_active":"2024-09-13T09:20:10.845257+0000","last_peered":"2024-09-13T09:20:10.845257+0000","last_clean":"2024-09-13T09:20:10.845257+0000","last_became_active":"2024-09-13T09:18:03.973314+0000","last_became_peered":"2024-09-13T09:18:03.973314+0000","last_unstale":"2024-09-13T09:20:10.845257+0000","last_undegraded":"2024-09-13T09:20:10.845257+0000","last_fullsized":"2024-09-13T09:20:10.845257+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.256457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012376119999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1399,"num_read_kb":26203,"num_write":287,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":167476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844907+0000","last_change":"2024-09-13T09:18:03.968679+0000","last_active":"2024-09-13T09:20:10.844907+0000","last_peered":"2024-09-13T09:20:10.844907+0000","last_clean":"2024-09-13T09:20:10.844907+0000","last_became_active":"2024-09-13T09:18:03.967133+0000","last_became_peered":"2024-09-13T09:18:03.967133+0000","last_unstale":"2024-09-13T09:20:10.844907+0000","last_undegraded":"2024-09-13T09:20:10.844907+0000","last_fullsized":"2024-09-13T09:20:10.844907+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:06.643934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"146'344","reported_seq":2268,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844102+0000","last_change":"2024-09-13T09:19:51.713872+0000","last_active":"2024-09-13T09:20:10.844102+0000","last_peered":"2024-09-13T09:20:10.844102+0000","last_clean":"2024-09-13T09:20:10.844102+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:20:10.844102+0000","last_undegraded":"2024-09-13T09:20:10.844102+0000","last_fullsized":"2024-09-13T09:20:10.844102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:00.161632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00109892,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2142,"num_read_kb":9020,"num_write":356,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"149'655","reported_seq":4751,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335893+0000","last_change":"2024-09-13T09:19:51.709444+0000","last_active":"2024-09-13T09:20:12.335893+0000","last_peered":"2024-09-13T09:20:12.335893+0000","last_clean":"2024-09-13T09:20:12.335893+0000","last_became_active":"2024-09-13T09:17:40.800027+0000","last_became_peered":"2024-09-13T09:17:40.800027+0000","last_unstale":"2024-09-13T09:20:12.335893+0000","last_undegraded":"2024-09-13T09:20:12.335893+0000","last_fullsized":"2024-09-13T09:20:12.335893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:10.182783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072429099999999995,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":5951,"num_read_kb":14391,"num_write":711,"num_write_kb":6974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":29248,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":199,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.843866+0000","last_change":"2024-09-13T09:18:03.976890+0000","last_active":"2024-09-13T09:20:10.843866+0000","last_peered":"2024-09-13T09:20:10.843866+0000","last_clean":"2024-09-13T09:20:10.843866+0000","last_became_active":"2024-09-13T09:18:03.970835+0000","last_became_peered":"2024-09-13T09:18:03.970835+0000","last_unstale":"2024-09-13T09:20:10.843866+0000","last_undegraded":"2024-09-13T09:20:10.843866+0000","last_fullsized":"2024-09-13T09:20:10.843866+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:05.189083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"147'561","reported_seq":2680,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845372+0000","last_change":"2024-09-13T09:19:51.709196+0000","last_active":"2024-09-13T09:20:10.845372+0000","last_peered":"2024-09-13T09:20:10.845372+0000","last_clean":"2024-09-13T09:20:10.845372+0000","last_became_active":"2024-09-13T09:18:03.970479+0000","last_became_peered":"2024-09-13T09:18:03.970479+0000","last_unstale":"2024-09-13T09:20:10.845372+0000","last_undegraded":"2024-09-13T09:20:10.845372+0000","last_fullsized":"2024-09-13T09:20:10.845372+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:06.229840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001031719,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5538,"num_read_kb":12570,"num_write":635,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1031390,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:09.833063+0000","last_change":"2024-09-13T09:20:02.757363+0000","last_active":"2024-09-13T09:20:09.833063+0000","last_peered":"2024-09-13T09:20:09.833063+0000","last_clean":"2024-09-13T09:20:09.833063+0000","last_became_active":"2024-09-13T09:20:02.756499+0000","last_became_peered":"2024-09-13T09:20:02.756499+0000","last_unstale":"2024-09-13T09:20:09.833063+0000","last_undegraded":"2024-09-13T09:20:09.833063+0000","last_fullsized":"2024-09-13T09:20:09.833063+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:36.829717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"147'483","reported_seq":3446,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816428+0000","last_change":"2024-09-13T09:19:51.724930+0000","last_active":"2024-09-13T09:20:11.816428+0000","last_peered":"2024-09-13T09:20:11.816428+0000","last_clean":"2024-09-13T09:20:11.816428+0000","last_became_active":"2024-09-13T09:17:40.793065+0000","last_became_peered":"2024-09-13T09:17:40.793065+0000","last_unstale":"2024-09-13T09:20:11.816428+0000","last_undegraded":"2024-09-13T09:20:11.816428+0000","last_fullsized":"2024-09-13T09:20:11.816428+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:38.586460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00175362,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4025,"num_read_kb":26901,"num_write":443,"num_write_kb":18254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":178,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336693+0000","last_change":"2024-09-13T09:18:03.968508+0000","last_active":"2024-09-13T09:20:12.336693+0000","last_peered":"2024-09-13T09:20:12.336693+0000","last_clean":"2024-09-13T09:20:12.336693+0000","last_became_active":"2024-09-13T09:18:03.968092+0000","last_became_peered":"2024-09-13T09:18:03.968092+0000","last_unstale":"2024-09-13T09:20:12.336693+0000","last_undegraded":"2024-09-13T09:20:12.336693+0000","last_fullsized":"2024-09-13T09:20:12.336693+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:39.649827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"148'782","reported_seq":2787,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844329+0000","last_change":"2024-09-13T09:19:51.712266+0000","last_active":"2024-09-13T09:20:10.844329+0000","last_peered":"2024-09-13T09:20:10.844329+0000","last_clean":"2024-09-13T09:20:10.844329+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:20:10.844329+0000","last_undegraded":"2024-09-13T09:20:10.844329+0000","last_fullsized":"2024-09-13T09:20:10.844329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:44.094850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012273519999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4598,"num_read_kb":13738,"num_write":1068,"num_write_kb":11812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":162,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.843866+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:20:10.843866+0000","last_peered":"2024-09-13T09:20:10.843866+0000","last_clean":"2024-09-13T09:20:10.843866+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:20:10.843866+0000","last_undegraded":"2024-09-13T09:20:10.843866+0000","last_fullsized":"2024-09-13T09:20:10.843866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":352,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336826+0000","last_change":"2024-09-13T09:17:43.803525+0000","last_active":"2024-09-13T09:20:12.336826+0000","last_peered":"2024-09-13T09:20:12.336826+0000","last_clean":"2024-09-13T09:20:12.336826+0000","last_became_active":"2024-09-13T09:17:43.802603+0000","last_became_peered":"2024-09-13T09:17:43.802603+0000","last_unstale":"2024-09-13T09:20:12.336826+0000","last_undegraded":"2024-09-13T09:20:12.336826+0000","last_fullsized":"2024-09-13T09:20:12.336826+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:07.561959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"147'650","reported_seq":2724,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844330+0000","last_change":"2024-09-13T09:19:51.713229+0000","last_active":"2024-09-13T09:20:10.844330+0000","last_peered":"2024-09-13T09:20:10.844330+0000","last_clean":"2024-09-13T09:20:10.844330+0000","last_became_active":"2024-09-13T09:18:03.974569+0000","last_became_peered":"2024-09-13T09:18:03.974569+0000","last_unstale":"2024-09-13T09:20:10.844330+0000","last_undegraded":"2024-09-13T09:20:10.844330+0000","last_fullsized":"2024-09-13T09:20:10.844330+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:43.479751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036429610000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4126,"num_read_kb":16863,"num_write":738,"num_write_kb":13412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":741496,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"148'539","reported_seq":2812,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335714+0000","last_change":"2024-09-13T09:19:51.709415+0000","last_active":"2024-09-13T09:20:12.335714+0000","last_peered":"2024-09-13T09:20:12.335714+0000","last_clean":"2024-09-13T09:20:12.335714+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:20:12.335714+0000","last_undegraded":"2024-09-13T09:20:12.335714+0000","last_fullsized":"2024-09-13T09:20:12.335714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:06.071033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070054600000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":11592,"num_write":623,"num_write_kb":5754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"148'505","reported_seq":3878,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902417+0000","last_change":"2024-09-13T09:19:51.826968+0000","last_active":"2024-09-13T09:20:08.902417+0000","last_peered":"2024-09-13T09:20:08.902417+0000","last_clean":"2024-09-13T09:20:08.902417+0000","last_became_active":"2024-09-13T09:19:34.891064+0000","last_became_peered":"2024-09-13T09:19:34.891064+0000","last_unstale":"2024-09-13T09:20:08.902417+0000","last_undegraded":"2024-09-13T09:20:08.902417+0000","last_fullsized":"2024-09-13T09:20:08.902417+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:11.410988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078961199999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":3950,"num_read_kb":15857,"num_write":467,"num_write_kb":7750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":4249683,"num_keys_recovered":143,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"149'429","reported_seq":1836,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844608+0000","last_change":"2024-09-13T09:19:51.712307+0000","last_active":"2024-09-13T09:20:10.844608+0000","last_peered":"2024-09-13T09:20:10.844608+0000","last_clean":"2024-09-13T09:20:10.844608+0000","last_became_active":"2024-09-13T09:17:40.801195+0000","last_became_peered":"2024-09-13T09:17:40.801195+0000","last_unstale":"2024-09-13T09:20:10.844608+0000","last_undegraded":"2024-09-13T09:20:10.844608+0000","last_fullsized":"2024-09-13T09:20:10.844608+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:00:07.773428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195911,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2626,"num_read_kb":8389,"num_write":522,"num_write_kb":11747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"149'683","reported_seq":1687,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337358+0000","last_change":"2024-09-13T09:19:51.722579+0000","last_active":"2024-09-13T09:20:12.337358+0000","last_peered":"2024-09-13T09:20:12.337358+0000","last_clean":"2024-09-13T09:20:12.337358+0000","last_became_active":"2024-09-13T09:19:49.736505+0000","last_became_peered":"2024-09-13T09:19:49.736505+0000","last_unstale":"2024-09-13T09:20:12.337358+0000","last_undegraded":"2024-09-13T09:20:12.337358+0000","last_fullsized":"2024-09-13T09:20:12.337358+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:30.617730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00218943,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2689,"num_read_kb":8298,"num_write":946,"num_write_kb":9158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":43,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"147'693","reported_seq":3232,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338076+0000","last_change":"2024-09-13T09:19:51.724802+0000","last_active":"2024-09-13T09:20:12.338076+0000","last_peered":"2024-09-13T09:20:12.338076+0000","last_clean":"2024-09-13T09:20:12.338076+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:20:12.338076+0000","last_undegraded":"2024-09-13T09:20:12.338076+0000","last_fullsized":"2024-09-13T09:20:12.338076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:25.411265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035729350000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4417,"num_read_kb":31654,"num_write":734,"num_write_kb":30325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"146'383","reported_seq":1377,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338412+0000","last_change":"2024-09-13T09:19:51.721061+0000","last_active":"2024-09-13T09:20:12.338412+0000","last_peered":"2024-09-13T09:20:12.338412+0000","last_clean":"2024-09-13T09:20:12.338412+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:20:12.338412+0000","last_undegraded":"2024-09-13T09:20:12.338412+0000","last_fullsized":"2024-09-13T09:20:12.338412+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:31.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068946100000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1301,"num_read_kb":10607,"num_write":406,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"148'980","reported_seq":4053,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338100+0000","last_change":"2024-09-13T09:19:51.722971+0000","last_active":"2024-09-13T09:20:12.338100+0000","last_peered":"2024-09-13T09:20:12.338100+0000","last_clean":"2024-09-13T09:20:12.338100+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:20:12.338100+0000","last_undegraded":"2024-09-13T09:20:12.338100+0000","last_fullsized":"2024-09-13T09:20:12.338100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:56.423075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024735249999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7129,"num_read_kb":22784,"num_write":1446,"num_write_kb":10311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"146'402","reported_seq":2308,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902107+0000","last_change":"2024-09-13T09:19:51.826873+0000","last_active":"2024-09-13T09:20:08.902107+0000","last_peered":"2024-09-13T09:20:08.902107+0000","last_clean":"2024-09-13T09:20:08.902107+0000","last_became_active":"2024-09-13T09:19:20.803495+0000","last_became_peered":"2024-09-13T09:19:20.803495+0000","last_unstale":"2024-09-13T09:20:08.902107+0000","last_undegraded":"2024-09-13T09:20:08.902107+0000","last_fullsized":"2024-09-13T09:20:08.902107+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:18.288761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072899800000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2117,"num_read_kb":10151,"num_write":496,"num_write_kb":12845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5102507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"148'549","reported_seq":3010,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336735+0000","last_change":"2024-09-13T09:19:51.722468+0000","last_active":"2024-09-13T09:20:12.336735+0000","last_peered":"2024-09-13T09:20:12.336735+0000","last_clean":"2024-09-13T09:20:12.336735+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:20:12.336735+0000","last_undegraded":"2024-09-13T09:20:12.336735+0000","last_fullsized":"2024-09-13T09:20:12.336735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:47.572839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020210359999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":14560,"num_write":713,"num_write_kb":15391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"146'438","reported_seq":4935,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.816485+0000","last_change":"2024-09-13T09:19:51.724223+0000","last_active":"2024-09-13T09:20:11.816485+0000","last_peered":"2024-09-13T09:20:11.816485+0000","last_clean":"2024-09-13T09:20:11.816485+0000","last_became_active":"2024-09-13T09:17:40.789499+0000","last_became_peered":"2024-09-13T09:17:40.789499+0000","last_unstale":"2024-09-13T09:20:11.816485+0000","last_undegraded":"2024-09-13T09:20:11.816485+0000","last_fullsized":"2024-09-13T09:20:11.816485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:13.395268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010644070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5803,"num_read_kb":44996,"num_write":398,"num_write_kb":17054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"147'421","reported_seq":3620,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337093+0000","last_change":"2024-09-13T09:19:51.723691+0000","last_active":"2024-09-13T09:20:12.337093+0000","last_peered":"2024-09-13T09:20:12.337093+0000","last_clean":"2024-09-13T09:20:12.337093+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:20:12.337093+0000","last_undegraded":"2024-09-13T09:20:12.337093+0000","last_fullsized":"2024-09-13T09:20:12.337093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:07.893848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003077476,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4122,"num_read_kb":18425,"num_write":408,"num_write_kb":9869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":152,"num_read":126915,"num_read_kb":555724,"num_write":19133,"num_write_kb":379149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1469,"num_bytes_recovered":28693609,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16684,"ondisk_log_size":16684,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":30095,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59380,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312460,"statfs":{"total":96636764160,"available":96575959040,"internally_reserved":0,"allocated":2068480,"data_stored":1548192,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":0.94799999999999995,"15min":0.88900000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":11.683,"5min":11.683,"15min":11.683},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.03,"15min":0.98199999999999998},"min":{"1min":0.48099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":11.843999999999999,"5min":11.843999999999999,"15min":11.843999999999999},"last":0.83799999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.86299999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.222,"5min":1.286,"15min":1.286},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.95199999999999996,"15min":0.98399999999999999},"min":{"1min":0.51200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.089,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":1.119}]},{"osd":2,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80000000000000004,"15min":0.80300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.153},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.94899999999999995,"15min":0.95899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.24,"5min":1.385,"15min":1.385},"last":1.0009999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.038,"15min":0.97399999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":12.012,"5min":12.012,"15min":12.012},"last":0.873},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.135,"15min":1.075},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":11.776999999999999,"5min":11.776999999999999,"15min":11.776999999999999},"last":1.28}]},{"osd":4,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":1.028,"15min":1.0609999999999999},"min":{"1min":0.44400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2969999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.4590000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.0649999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.073,"15min":1.071},"min":{"1min":0.61199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.3540000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.121,"15min":1.145},"min":{"1min":0.51900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":3.125,"15min":3.125},"last":1.2190000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.1950000000000001,"15min":1.2330000000000001},"min":{"1min":0.59199999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.667,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.109,"15min":1.119},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.498,"5min":1.996,"15min":1.996},"last":1.3100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41272,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330568,"statfs":{"total":96636764160,"available":96594501632,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.438,"5min":0.314,"15min":0.314},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.877},"min":{"1min":0.40899999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3580000000000001,"5min":2.222,"15min":2.222},"last":0.59699999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.83399999999999996,"15min":0.81200000000000006},"min":{"1min":0.50800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.98199999999999998,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.607,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.65700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4319999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.98499999999999999,"15min":1.0029999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.613,"5min":2.508,"15min":2.508},"last":0.70999999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.66900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.55,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.878},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.2450000000000001,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.83799999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.95199999999999996,"15min":0.97399999999999998},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2669999999999999,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.008,"15min":1},"min":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5800000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.92900000000000005}]},{"osd":5,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.92300000000000004,"15min":0.90000000000000002},"min":{"1min":0.58999999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":1.02},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.57599999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.8120000000000001,"5min":2.286,"15min":2.286},"last":1.0720000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.093,"15min":1.087},"min":{"1min":0.621,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.611,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.079,"15min":1.0489999999999999},"min":{"1min":0.69899999999999995,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.1719999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55236,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316604,"statfs":{"total":96636764160,"available":96580202496,"internally_reserved":0,"allocated":2015232,"data_stored":957843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":6.3970000000000002,"15min":7.3419999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1739999999999999,"5min":132.32900000000001,"15min":132.32900000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.236,"5min":1.335,"15min":1.335},"last":1.0589999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.86499999999999999,"15min":0.88500000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96399999999999997,"5min":1.946,"15min":1.946},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.998,"15min":1.0209999999999999},"min":{"1min":0.56899999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1579999999999999,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.71199999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.85099999999999998,"15min":0.84699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1220000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.88800000000000001,"15min":0.89700000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1719999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.98199999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.93799999999999994,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2849999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.028},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0009999999999999,"15min":1.006},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4419999999999999,"5min":1.476,"15min":1.476},"last":0.77500000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86899999999999999,"15min":0.86499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3140000000000001,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":6.8070000000000004,"15min":7.7770000000000001},"min":{"1min":0.47699999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.544,"5min":132.23599999999999,"15min":132.23599999999999},"last":1.135}]},{"osd":6,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.016,"15min":1.0329999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3160000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.079}]},{"osd":7,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.075,"15min":1.075},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.452,"5min":1.655,"15min":1.655},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1279999999999999,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6379999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":0.94999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51136,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320704,"statfs":{"total":96636764160,"available":96584400896,"internally_reserved":0,"allocated":2015232,"data_stored":957843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3560000000000001,"5min":1.357,"15min":1.357},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.94599999999999995,"15min":0.94799999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.331}]},{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91400000000000003,"15min":0.92600000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.81100000000000005,"15min":0.81999999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.877},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.006,"5min":0.88,"15min":0.84299999999999997},"min":{"1min":0.56999999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.111}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86899999999999999,"15min":0.85399999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3300000000000001,"5min":1.575,"15min":1.575},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.069,"5min":1.0249999999999999,"15min":0.97799999999999998},"min":{"1min":0.628,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0899999999999999,"5min":4.7389999999999999,"15min":4.7389999999999999},"last":1.0580000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.45400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.3720000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0329999999999999},"min":{"1min":0.53500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.077,"15min":1.0629999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.4119999999999999,"5min":2.4119999999999999,"15min":2.4119999999999999},"last":1.5720000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.034,"15min":1.026},"min":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5609999999999999,"5min":1.597,"15min":1.597},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.125,"15min":1.131},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.2769999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77372,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94294468,"statfs":{"total":96636764160,"available":96557535232,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.313,"5min":0.215,"15min":0.215},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.62},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.96399999999999997,"5min":1.625,"15min":1.625},"last":0.754}]},{"osd":1,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.28799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.80600000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.36899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.425,"5min":1.544,"15min":1.544},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.755,"15min":0.76200000000000001},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.98199999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89200000000000002,"15min":0.90600000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.25,"5min":1.669,"15min":1.669},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.93300000000000005,"15min":0.94199999999999995},"min":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.296,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.872}]},{"osd":5,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.97199999999999998},"min":{"1min":0.59999999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.5580000000000001,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.92400000000000004,"15min":0.93200000000000005},"min":{"1min":0.63,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3380000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.90400000000000003}]},{"osd":6,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.628,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.47,"5min":1.748,"15min":1.748},"last":1.038},{"interface":"front","average":{"1min":1.024,"5min":1.0860000000000001,"15min":1.0960000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5620000000000001,"5min":2.484,"15min":2.484},"last":1.1950000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.998,"15min":1.006},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.385,"5min":1.806,"15min":1.806},"last":1.01},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.665,"15min":1.772},"min":{"1min":0.61299999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.3879999999999999,"5min":15.4,"15min":15.4},"last":1.0920000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41272,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330568,"statfs":{"total":96636764160,"available":96594501632,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.621},"min":{"1min":0.247,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1899999999999999,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.62},{"interface":"front","average":{"1min":0.749,"5min":0.72499999999999998,"15min":0.70499999999999996},"min":{"1min":0.28399999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.671,"5min":2.8490000000000002,"15min":2.8490000000000002},"last":0.93400000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.79000000000000004,"15min":0.79300000000000004},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.337,"5min":1.907,"15min":1.907},"last":1.137}]},{"osd":3,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.77600000000000002,"15min":0.74399999999999999},"min":{"1min":0.48999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.256,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.83899999999999997,"15min":0.871},"min":{"1min":0.34000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.47,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.97399999999999998,"15min":0.96999999999999997},"min":{"1min":0.76200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4470000000000001,"5min":1.782,"15min":1.782},"last":1.27},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.83499999999999996,"15min":0.81699999999999995},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.97999999999999998,"15min":0.96799999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0329999999999999,"15min":1.0289999999999999},"min":{"1min":0.57199999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5580000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.3180000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.55600000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.012,"15min":1.0049999999999999},"min":{"1min":0.623,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.923,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.88100000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.121,"15min":1.133},"min":{"1min":0.68899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.573,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.226},{"interface":"front","average":{"1min":1.071,"5min":1.0049999999999999,"15min":1.0029999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5620000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.84799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":2019328,"data_stored":957862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.70599999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.429,"5min":1.484,"15min":1.484},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65500000000000003,"15min":0.67100000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.091,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.35299999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.212,"5min":1.264,"15min":1.264},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76000000000000001,"15min":0.76700000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.92900000000000005,"15min":0.95899999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3029999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.98299999999999998,"15min":1.0129999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.22,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.0760000000000001,"15min":1.04},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":8.3460000000000001,"5min":8.3460000000000001,"15min":8.3460000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.998,"15min":1.022},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2769999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.024,"15min":1.0489999999999999},"min":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.339,"5min":1.647,"15min":1.647},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.012,"15min":1.0309999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.389,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.03,"15min":1.052},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.345,"5min":1.629,"15min":1.629},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.095,"15min":1.119},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.411,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.90800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":2052096,"data_stored":1548192,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.316,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.325,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.625,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.006,"5min":1.284,"15min":1.284},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92600000000000005,"5min":1.5,"15min":1.5},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.39000000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.129,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.72499999999999998,"15min":0.69899999999999995},"min":{"1min":0.42699999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.452,"5min":2.242,"15min":2.242},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.78900000000000003},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2250000000000001,"5min":1.528,"15min":1.528},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.89200000000000002,"15min":0.877},"min":{"1min":0.53700000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0309999999999999,"15min":0.96599999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.484,"5min":9.5649999999999995,"15min":9.5649999999999995},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0469999999999999,"15min":0.95799999999999996},"min":{"1min":0.67900000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.375,"5min":9.6120000000000001,"15min":9.6120000000000001},"last":0.75},{"interface":"front","average":{"1min":1.036,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.419,"5min":1.476,"15min":1.476},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0620000000000001,"15min":1.085},"min":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4279999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.996,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.482,"5min":1.516,"15min":1.516},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0229999999999999,"15min":1.0309999999999999},"min":{"1min":0.58799999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.5669999999999999,"5min":1.617,"15min":1.617},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.995,"5min":1.1379999999999999,"15min":1.0820000000000001},"min":{"1min":0.67300000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.546,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":0.78600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:20:16.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:20:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:20:17.244 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":167,"fsid":"f0889cb1-03d6-4b81-88c9-6141846c6033","created":"2024-09-13T09:16:56.964982+0000","modified":"2024-09-13T09:20:11.848833+0000","last_up_change":"2024-09-13T09:17:08.368378+0000","last_in_change":"2024-09-13T09:19:16.749126+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:17:10.140920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:17:15.775544+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T09:17:25.804931+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":81,"snap_epoch":149,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"786bdfc4-cc25-4167-aa5b-3737cfeb4ac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6801","nonce":3549111487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6803","nonce":3549111487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6807","nonce":3549111487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3549111487},{"type":"v1","addr":"172.21.15.32:6805","nonce":3549111487}]},"public_addr":"172.21.15.32:6801/3549111487","cluster_addr":"172.21.15.32:6803/3549111487","heartbeat_back_addr":"172.21.15.32:6807/3549111487","heartbeat_front_addr":"172.21.15.32:6805/3549111487","state":["exists","up"]},{"osd":1,"uuid":"ca6189ab-90dd-4e9d-a85f-092186153b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6810","nonce":1470371899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6815","nonce":1470371899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6823","nonce":1470371899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1470371899},{"type":"v1","addr":"172.21.15.32:6820","nonce":1470371899}]},"public_addr":"172.21.15.32:6810/1470371899","cluster_addr":"172.21.15.32:6815/1470371899","heartbeat_back_addr":"172.21.15.32:6823/1470371899","heartbeat_front_addr":"172.21.15.32:6820/1470371899","state":["exists","up"]},{"osd":2,"uuid":"c688a5bf-2512-4c3a-b9ef-e952d4d0fb6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6825","nonce":2485123706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6827","nonce":2485123706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6831","nonce":2485123706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2485123706},{"type":"v1","addr":"172.21.15.32:6829","nonce":2485123706}]},"public_addr":"172.21.15.32:6825/2485123706","cluster_addr":"172.21.15.32:6827/2485123706","heartbeat_back_addr":"172.21.15.32:6831/2485123706","heartbeat_front_addr":"172.21.15.32:6829/2485123706","state":["exists","up"]},{"osd":3,"uuid":"5faf5fc6-4260-4ee2-8b95-3d8bb2a3a218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6811","nonce":1009890408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6814","nonce":1009890408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6821","nonce":1009890408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1009890408},{"type":"v1","addr":"172.21.15.32:6818","nonce":1009890408}]},"public_addr":"172.21.15.32:6811/1009890408","cluster_addr":"172.21.15.32:6814/1009890408","heartbeat_back_addr":"172.21.15.32:6821/1009890408","heartbeat_front_addr":"172.21.15.32:6818/1009890408","state":["exists","up"]},{"osd":4,"uuid":"39f42e87-29dd-42cc-9ef6-e0074f1794e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6823","nonce":2723023689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6825","nonce":2723023689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6831","nonce":2723023689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":2723023689},{"type":"v1","addr":"172.21.15.46:6828","nonce":2723023689}]},"public_addr":"172.21.15.46:6823/2723023689","cluster_addr":"172.21.15.46:6825/2723023689","heartbeat_back_addr":"172.21.15.46:6831/2723023689","heartbeat_front_addr":"172.21.15.46:6828/2723023689","state":["exists","up"]},{"osd":5,"uuid":"cf8dd701-9543-4bdd-a6f6-0b56210f00c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6809","nonce":2955019640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6812","nonce":2955019640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2955019640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2955019640},{"type":"v1","addr":"172.21.15.46:6815","nonce":2955019640}]},"public_addr":"172.21.15.46:6809/2955019640","cluster_addr":"172.21.15.46:6812/2955019640","heartbeat_back_addr":"172.21.15.46:6819/2955019640","heartbeat_front_addr":"172.21.15.46:6815/2955019640","state":["exists","up"]},{"osd":6,"uuid":"dd080b88-3292-4a34-8dbd-645683d8fe14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6813","nonce":916160151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6818","nonce":916160151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6829","nonce":916160151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":916160151},{"type":"v1","addr":"172.21.15.46:6822","nonce":916160151}]},"public_addr":"172.21.15.46:6813/916160151","cluster_addr":"172.21.15.46:6818/916160151","heartbeat_back_addr":"172.21.15.46:6829/916160151","heartbeat_front_addr":"172.21.15.46:6822/916160151","state":["exists","up"]},{"osd":7,"uuid":"c6bca54b-e982-4d6a-9ae9-057ee3881a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6801","nonce":3477167689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6803","nonce":3477167689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6807","nonce":3477167689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3477167689},{"type":"v1","addr":"172.21.15.46:6805","nonce":3477167689}]},"public_addr":"172.21.15.46:6801/3477167689","cluster_addr":"172.21.15.46:6803/3477167689","heartbeat_back_addr":"172.21.15.46:6807/3477167689","heartbeat_front_addr":"172.21.15.46:6805/3477167689","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.247113+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.166872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.451769+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.458130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.466129+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.444828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.425145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:17:05.484371+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:20:18.264 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T09:20:18.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:20:18.429 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:20:18.429 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:20:18.429 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:20:18.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T09:20:18.735 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T09:20:18.754 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T09:20:18.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:20:18.901 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:20:18.902 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:20:18.902 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:20:18.919 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T09:20:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T09:20:19.226 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T09:20:19.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:20:19.377 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:20:19.378 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:20:19.378 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:20:19.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T09:20:19.677 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T09:20:19.694 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T09:20:19.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:20:19.846 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:20:19.846 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:20:19.846 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:20:19.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T09:20:20.152 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T09:20:20.172 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T09:20:20.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:20:20.331 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:20:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:20:20.331 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:20:20.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T09:20:20.638 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T09:20:20.657 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T09:20:20.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:20:20.806 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:20:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:20:20.806 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:20:20.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T09:20:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T09:20:21.135 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T09:20:21.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:20:21.285 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:20:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:20:21.285 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:20:21.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T09:20:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T09:20:21.598 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T09:20:21.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:20:21.750 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-13T09:20:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:20:21.750 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-13T09:20:21.765 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T09:20:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T09:20:22.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:20:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:20:22.376 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:20:22.393 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-13T09:20:22.108263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":152,"num_read":127031,"num_read_kb":555824,"num_write":19294,"num_write_kb":381567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1476,"num_bytes_recovered":29874402,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16804,"ondisk_log_size":16804,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":391880,"kb_used_data":15824,"kb_used_omap":12,"kb_used_meta":375923,"kb_avail":754582840,"statfs":{"total":773094113280,"available":772692828160,"internally_reserved":0,"allocated":16203776,"data_stored":8843404,"data_compressed":19717,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12744,"internal_metadata":384945720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.249925"},"pg_stats":[{"pgid":"3.1f","version":"149'406","reported_seq":1884,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.862407+0000","last_change":"2024-09-13T09:19:51.725659+0000","last_active":"2024-09-13T09:20:11.862407+0000","last_peered":"2024-09-13T09:20:11.862407+0000","last_clean":"2024-09-13T09:20:11.862407+0000","last_became_active":"2024-09-13T09:17:40.791187+0000","last_became_peered":"2024-09-13T09:17:40.791187+0000","last_unstale":"2024-09-13T09:20:11.862407+0000","last_undegraded":"2024-09-13T09:20:11.862407+0000","last_fullsized":"2024-09-13T09:20:11.862407+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001951139,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2539,"num_read_kb":13964,"num_write":479,"num_write_kb":12323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"146'640","reported_seq":3283,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844414+0000","last_change":"2024-09-13T09:19:51.712843+0000","last_active":"2024-09-13T09:20:10.844414+0000","last_peered":"2024-09-13T09:20:10.844414+0000","last_clean":"2024-09-13T09:20:10.844414+0000","last_became_active":"2024-09-13T09:19:20.802379+0000","last_became_peered":"2024-09-13T09:19:20.802379+0000","last_unstale":"2024-09-13T09:20:10.844414+0000","last_undegraded":"2024-09-13T09:20:10.844414+0000","last_fullsized":"2024-09-13T09:20:10.844414+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:20.952086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001400552,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3630,"num_read_kb":10038,"num_write":856,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":8249616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"146'488","reported_seq":3762,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844008+0000","last_change":"2024-09-13T09:19:51.714853+0000","last_active":"2024-09-13T09:20:10.844008+0000","last_peered":"2024-09-13T09:20:10.844008+0000","last_clean":"2024-09-13T09:20:10.844008+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:20:10.844008+0000","last_undegraded":"2024-09-13T09:20:10.844008+0000","last_fullsized":"2024-09-13T09:20:10.844008+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:20.509991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021011329999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5768,"num_read_kb":20052,"num_write":439,"num_write_kb":19543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"149'793","reported_seq":4565,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.843864+0000","last_change":"2024-09-13T09:19:51.715407+0000","last_active":"2024-09-13T09:20:10.843864+0000","last_peered":"2024-09-13T09:20:10.843864+0000","last_clean":"2024-09-13T09:20:10.843864+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:20:10.843864+0000","last_undegraded":"2024-09-13T09:20:10.843864+0000","last_fullsized":"2024-09-13T09:20:10.843864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:05.002231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025684319999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":7779,"num_read_kb":18614,"num_write":881,"num_write_kb":10036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"148'531","reported_seq":4432,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.776929+0000","last_change":"2024-09-13T09:19:51.725489+0000","last_active":"2024-09-13T09:20:12.776929+0000","last_peered":"2024-09-13T09:20:12.776929+0000","last_clean":"2024-09-13T09:20:12.776929+0000","last_became_active":"2024-09-13T09:17:40.799618+0000","last_became_peered":"2024-09-13T09:17:40.799618+0000","last_unstale":"2024-09-13T09:20:12.776929+0000","last_undegraded":"2024-09-13T09:20:12.776929+0000","last_fullsized":"2024-09-13T09:20:12.776929+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001823089,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5766,"num_read_kb":19616,"num_write":659,"num_write_kb":9141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":65341,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"146'344","reported_seq":3288,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335538+0000","last_change":"2024-09-13T09:19:51.709691+0000","last_active":"2024-09-13T09:20:12.335538+0000","last_peered":"2024-09-13T09:20:12.335538+0000","last_clean":"2024-09-13T09:20:12.335538+0000","last_became_active":"2024-09-13T09:18:03.973063+0000","last_became_peered":"2024-09-13T09:18:03.973063+0000","last_unstale":"2024-09-13T09:20:12.335538+0000","last_undegraded":"2024-09-13T09:20:12.335538+0000","last_fullsized":"2024-09-13T09:20:12.335538+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:29.911994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082503800000000005,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3197,"num_read_kb":8961,"num_write":344,"num_write_kb":6710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":889520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'715","reported_seq":4250,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844995+0000","last_change":"2024-09-13T09:19:51.709213+0000","last_active":"2024-09-13T09:20:10.844995+0000","last_peered":"2024-09-13T09:20:10.844995+0000","last_clean":"2024-09-13T09:20:10.844995+0000","last_became_active":"2024-09-13T09:18:06.975016+0000","last_became_peered":"2024-09-13T09:18:06.975016+0000","last_unstale":"2024-09-13T09:20:10.844995+0000","last_undegraded":"2024-09-13T09:20:10.844995+0000","last_fullsized":"2024-09-13T09:20:10.844995+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:42:30.489567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010502549999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5493,"num_read_kb":28565,"num_write":1011,"num_write_kb":11903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":439303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"149'217","reported_seq":2561,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844055+0000","last_change":"2024-09-13T09:19:51.710337+0000","last_active":"2024-09-13T09:20:10.844055+0000","last_peered":"2024-09-13T09:20:10.844055+0000","last_clean":"2024-09-13T09:20:10.844055+0000","last_became_active":"2024-09-13T09:17:40.797607+0000","last_became_peered":"2024-09-13T09:17:40.797607+0000","last_unstale":"2024-09-13T09:20:10.844055+0000","last_undegraded":"2024-09-13T09:20:10.844055+0000","last_fullsized":"2024-09-13T09:20:10.844055+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:09.025976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011389530000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2065,"num_read_kb":17454,"num_write":181,"num_write_kb":10975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":42856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"148'442","reported_seq":2302,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.862629+0000","last_change":"2024-09-13T09:19:51.725896+0000","last_active":"2024-09-13T09:20:11.862629+0000","last_peered":"2024-09-13T09:20:11.862629+0000","last_clean":"2024-09-13T09:20:11.862629+0000","last_became_active":"2024-09-13T09:17:43.809533+0000","last_became_peered":"2024-09-13T09:17:43.809533+0000","last_unstale":"2024-09-13T09:20:11.862629+0000","last_undegraded":"2024-09-13T09:20:11.862629+0000","last_fullsized":"2024-09-13T09:20:11.862629+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018671040000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3080,"num_read_kb":24506,"num_write":379,"num_write_kb":12955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":16384,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"148'272","reported_seq":2284,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844180+0000","last_change":"2024-09-13T09:19:51.714263+0000","last_active":"2024-09-13T09:20:10.844180+0000","last_peered":"2024-09-13T09:20:10.844180+0000","last_clean":"2024-09-13T09:20:10.844180+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:20:10.844180+0000","last_undegraded":"2024-09-13T09:20:10.844180+0000","last_fullsized":"2024-09-13T09:20:10.844180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:29.849557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001510742,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1901,"num_read_kb":15086,"num_write":247,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"147'371","reported_seq":2968,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337807+0000","last_change":"2024-09-13T09:19:51.723974+0000","last_active":"2024-09-13T09:20:12.337807+0000","last_peered":"2024-09-13T09:20:12.337807+0000","last_clean":"2024-09-13T09:20:12.337807+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:20:12.337807+0000","last_undegraded":"2024-09-13T09:20:12.337807+0000","last_fullsized":"2024-09-13T09:20:12.337807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:10.717628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029658509999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3481,"num_read_kb":18288,"num_write":412,"num_write_kb":15933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"142'580","reported_seq":3079,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336038+0000","last_change":"2024-09-13T09:19:51.709951+0000","last_active":"2024-09-13T09:20:12.336038+0000","last_peered":"2024-09-13T09:20:12.336038+0000","last_clean":"2024-09-13T09:20:12.336038+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:20:12.336038+0000","last_undegraded":"2024-09-13T09:20:12.336038+0000","last_fullsized":"2024-09-13T09:20:12.336038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:42.341412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001048134,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3471,"num_read_kb":17629,"num_write":764,"num_write_kb":18883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902086+0000","last_change":"2024-09-13T09:19:17.796958+0000","last_active":"2024-09-13T09:20:08.902086+0000","last_peered":"2024-09-13T09:20:08.902086+0000","last_clean":"2024-09-13T09:20:08.902086+0000","last_became_active":"2024-09-13T09:19:17.796177+0000","last_became_peered":"2024-09-13T09:19:17.796177+0000","last_unstale":"2024-09-13T09:20:08.902086+0000","last_undegraded":"2024-09-13T09:20:08.902086+0000","last_fullsized":"2024-09-13T09:20:08.902086+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:59.576027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"149'450","reported_seq":1830,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.861539+0000","last_change":"2024-09-13T09:19:51.725161+0000","last_active":"2024-09-13T09:20:11.861539+0000","last_peered":"2024-09-13T09:20:11.861539+0000","last_clean":"2024-09-13T09:20:11.861539+0000","last_became_active":"2024-09-13T09:19:14.748293+0000","last_became_peered":"2024-09-13T09:19:14.748293+0000","last_unstale":"2024-09-13T09:20:11.861539+0000","last_undegraded":"2024-09-13T09:20:11.861539+0000","last_fullsized":"2024-09-13T09:20:11.861539+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016656990000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2578,"num_read_kb":17393,"num_write":357,"num_write_kb":10507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":7471802,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":145,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.791561+0000","last_change":"2024-09-13T09:19:17.770210+0000","last_active":"2024-09-13T09:20:12.791561+0000","last_peered":"2024-09-13T09:20:12.791561+0000","last_clean":"2024-09-13T09:20:12.791561+0000","last_became_active":"2024-09-13T09:19:17.769570+0000","last_became_peered":"2024-09-13T09:19:17.769570+0000","last_unstale":"2024-09-13T09:20:12.791561+0000","last_undegraded":"2024-09-13T09:20:12.791561+0000","last_fullsized":"2024-09-13T09:20:12.791561+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:37.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"147'561","reported_seq":3012,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845458+0000","last_change":"2024-09-13T09:19:51.712759+0000","last_active":"2024-09-13T09:20:10.845458+0000","last_peered":"2024-09-13T09:20:10.845458+0000","last_clean":"2024-09-13T09:20:10.845458+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:20:10.845458+0000","last_undegraded":"2024-09-13T09:20:10.845458+0000","last_fullsized":"2024-09-13T09:20:10.845458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:51.225828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015869739999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4402,"num_read_kb":8559,"num_write":727,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338299+0000","last_change":"2024-09-13T09:17:40.794662+0000","last_active":"2024-09-13T09:20:12.338299+0000","last_peered":"2024-09-13T09:20:12.338299+0000","last_clean":"2024-09-13T09:20:12.338299+0000","last_became_active":"2024-09-13T09:17:40.793882+0000","last_became_peered":"2024-09-13T09:17:40.793882+0000","last_unstale":"2024-09-13T09:20:12.338299+0000","last_undegraded":"2024-09-13T09:20:12.338299+0000","last_fullsized":"2024-09-13T09:20:12.338299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:22.922753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"148'377","reported_seq":1622,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845257+0000","last_change":"2024-09-13T09:19:51.709972+0000","last_active":"2024-09-13T09:20:10.845257+0000","last_peered":"2024-09-13T09:20:10.845257+0000","last_clean":"2024-09-13T09:20:10.845257+0000","last_became_active":"2024-09-13T09:18:03.973314+0000","last_became_peered":"2024-09-13T09:18:03.973314+0000","last_unstale":"2024-09-13T09:20:10.845257+0000","last_undegraded":"2024-09-13T09:20:10.845257+0000","last_fullsized":"2024-09-13T09:20:10.845257+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:28.256457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012376119999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1399,"num_read_kb":26203,"num_write":287,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":167476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844907+0000","last_change":"2024-09-13T09:18:03.968679+0000","last_active":"2024-09-13T09:20:10.844907+0000","last_peered":"2024-09-13T09:20:10.844907+0000","last_clean":"2024-09-13T09:20:10.844907+0000","last_became_active":"2024-09-13T09:18:03.967133+0000","last_became_peered":"2024-09-13T09:18:03.967133+0000","last_unstale":"2024-09-13T09:20:10.844907+0000","last_undegraded":"2024-09-13T09:20:10.844907+0000","last_fullsized":"2024-09-13T09:20:10.844907+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:06.643934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"146'344","reported_seq":2268,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844102+0000","last_change":"2024-09-13T09:19:51.713872+0000","last_active":"2024-09-13T09:20:10.844102+0000","last_peered":"2024-09-13T09:20:10.844102+0000","last_clean":"2024-09-13T09:20:10.844102+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:20:10.844102+0000","last_undegraded":"2024-09-13T09:20:10.844102+0000","last_fullsized":"2024-09-13T09:20:10.844102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:00.161632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00109892,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2142,"num_read_kb":9020,"num_write":356,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"149'655","reported_seq":4751,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335893+0000","last_change":"2024-09-13T09:19:51.709444+0000","last_active":"2024-09-13T09:20:12.335893+0000","last_peered":"2024-09-13T09:20:12.335893+0000","last_clean":"2024-09-13T09:20:12.335893+0000","last_became_active":"2024-09-13T09:17:40.800027+0000","last_became_peered":"2024-09-13T09:17:40.800027+0000","last_unstale":"2024-09-13T09:20:12.335893+0000","last_undegraded":"2024-09-13T09:20:12.335893+0000","last_fullsized":"2024-09-13T09:20:12.335893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:10.182783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072429099999999995,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":5951,"num_read_kb":14391,"num_write":711,"num_write_kb":6974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":29248,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":199,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.843866+0000","last_change":"2024-09-13T09:18:03.976890+0000","last_active":"2024-09-13T09:20:10.843866+0000","last_peered":"2024-09-13T09:20:10.843866+0000","last_clean":"2024-09-13T09:20:10.843866+0000","last_became_active":"2024-09-13T09:18:03.970835+0000","last_became_peered":"2024-09-13T09:18:03.970835+0000","last_unstale":"2024-09-13T09:20:10.843866+0000","last_undegraded":"2024-09-13T09:20:10.843866+0000","last_fullsized":"2024-09-13T09:20:10.843866+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:05.189083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"147'561","reported_seq":2680,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.845372+0000","last_change":"2024-09-13T09:19:51.709196+0000","last_active":"2024-09-13T09:20:10.845372+0000","last_peered":"2024-09-13T09:20:10.845372+0000","last_clean":"2024-09-13T09:20:10.845372+0000","last_became_active":"2024-09-13T09:18:03.970479+0000","last_became_peered":"2024-09-13T09:18:03.970479+0000","last_unstale":"2024-09-13T09:20:10.845372+0000","last_undegraded":"2024-09-13T09:20:10.845372+0000","last_fullsized":"2024-09-13T09:20:10.845372+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:06.229840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001031719,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5538,"num_read_kb":12570,"num_write":635,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1031390,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:09.833063+0000","last_change":"2024-09-13T09:20:02.757363+0000","last_active":"2024-09-13T09:20:09.833063+0000","last_peered":"2024-09-13T09:20:09.833063+0000","last_clean":"2024-09-13T09:20:09.833063+0000","last_became_active":"2024-09-13T09:20:02.756499+0000","last_became_peered":"2024-09-13T09:20:02.756499+0000","last_unstale":"2024-09-13T09:20:09.833063+0000","last_undegraded":"2024-09-13T09:20:09.833063+0000","last_fullsized":"2024-09-13T09:20:09.833063+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:36.829717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"147'483","reported_seq":3447,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.861261+0000","last_change":"2024-09-13T09:19:51.724930+0000","last_active":"2024-09-13T09:20:11.861261+0000","last_peered":"2024-09-13T09:20:11.861261+0000","last_clean":"2024-09-13T09:20:11.861261+0000","last_became_active":"2024-09-13T09:17:40.793065+0000","last_became_peered":"2024-09-13T09:17:40.793065+0000","last_unstale":"2024-09-13T09:20:11.861261+0000","last_undegraded":"2024-09-13T09:20:11.861261+0000","last_fullsized":"2024-09-13T09:20:11.861261+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00175362,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4025,"num_read_kb":26901,"num_write":443,"num_write_kb":18254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":178,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336693+0000","last_change":"2024-09-13T09:18:03.968508+0000","last_active":"2024-09-13T09:20:12.336693+0000","last_peered":"2024-09-13T09:20:12.336693+0000","last_clean":"2024-09-13T09:20:12.336693+0000","last_became_active":"2024-09-13T09:18:03.968092+0000","last_became_peered":"2024-09-13T09:18:03.968092+0000","last_unstale":"2024-09-13T09:20:12.336693+0000","last_undegraded":"2024-09-13T09:20:12.336693+0000","last_fullsized":"2024-09-13T09:20:12.336693+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:39.649827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"148'782","reported_seq":2787,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844329+0000","last_change":"2024-09-13T09:19:51.712266+0000","last_active":"2024-09-13T09:20:10.844329+0000","last_peered":"2024-09-13T09:20:10.844329+0000","last_clean":"2024-09-13T09:20:10.844329+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:20:10.844329+0000","last_undegraded":"2024-09-13T09:20:10.844329+0000","last_fullsized":"2024-09-13T09:20:10.844329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:44.094850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012273519999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4598,"num_read_kb":13738,"num_write":1068,"num_write_kb":11812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":162,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.843866+0000","last_change":"2024-09-13T09:17:19.508461+0000","last_active":"2024-09-13T09:20:10.843866+0000","last_peered":"2024-09-13T09:20:10.843866+0000","last_clean":"2024-09-13T09:20:10.843866+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:20:10.843866+0000","last_undegraded":"2024-09-13T09:20:10.843866+0000","last_fullsized":"2024-09-13T09:20:10.843866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:16.469202+0000","last_clean_scrub_stamp":"2024-09-13T09:17:16.469202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:58:28.602596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":352,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336826+0000","last_change":"2024-09-13T09:17:43.803525+0000","last_active":"2024-09-13T09:20:12.336826+0000","last_peered":"2024-09-13T09:20:12.336826+0000","last_clean":"2024-09-13T09:20:12.336826+0000","last_became_active":"2024-09-13T09:17:43.802603+0000","last_became_peered":"2024-09-13T09:17:43.802603+0000","last_unstale":"2024-09-13T09:20:12.336826+0000","last_undegraded":"2024-09-13T09:20:12.336826+0000","last_fullsized":"2024-09-13T09:20:12.336826+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:10.373878+0000","last_clean_scrub_stamp":"2024-09-13T09:17:10.373878+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:07.561959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"147'650","reported_seq":2724,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844330+0000","last_change":"2024-09-13T09:19:51.713229+0000","last_active":"2024-09-13T09:20:10.844330+0000","last_peered":"2024-09-13T09:20:10.844330+0000","last_clean":"2024-09-13T09:20:10.844330+0000","last_became_active":"2024-09-13T09:18:03.974569+0000","last_became_peered":"2024-09-13T09:18:03.974569+0000","last_unstale":"2024-09-13T09:20:10.844330+0000","last_undegraded":"2024-09-13T09:20:10.844330+0000","last_fullsized":"2024-09-13T09:20:10.844330+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:43.479751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036429610000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4126,"num_read_kb":16863,"num_write":738,"num_write_kb":13412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":741496,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"148'539","reported_seq":2812,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.335714+0000","last_change":"2024-09-13T09:19:51.709415+0000","last_active":"2024-09-13T09:20:12.335714+0000","last_peered":"2024-09-13T09:20:12.335714+0000","last_clean":"2024-09-13T09:20:12.335714+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:20:12.335714+0000","last_undegraded":"2024-09-13T09:20:12.335714+0000","last_fullsized":"2024-09-13T09:20:12.335714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:06.071033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070054600000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":11592,"num_write":623,"num_write_kb":5754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"148'505","reported_seq":3878,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902417+0000","last_change":"2024-09-13T09:19:51.826968+0000","last_active":"2024-09-13T09:20:08.902417+0000","last_peered":"2024-09-13T09:20:08.902417+0000","last_clean":"2024-09-13T09:20:08.902417+0000","last_became_active":"2024-09-13T09:19:34.891064+0000","last_became_peered":"2024-09-13T09:19:34.891064+0000","last_unstale":"2024-09-13T09:20:08.902417+0000","last_undegraded":"2024-09-13T09:20:08.902417+0000","last_fullsized":"2024-09-13T09:20:08.902417+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:11.410988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078961199999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":3950,"num_read_kb":15857,"num_write":467,"num_write_kb":7750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":4249683,"num_keys_recovered":143,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"149'429","reported_seq":1836,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T09:20:10.844608+0000","last_change":"2024-09-13T09:19:51.712307+0000","last_active":"2024-09-13T09:20:10.844608+0000","last_peered":"2024-09-13T09:20:10.844608+0000","last_clean":"2024-09-13T09:20:10.844608+0000","last_became_active":"2024-09-13T09:17:40.801195+0000","last_became_peered":"2024-09-13T09:17:40.801195+0000","last_unstale":"2024-09-13T09:20:10.844608+0000","last_undegraded":"2024-09-13T09:20:10.844608+0000","last_fullsized":"2024-09-13T09:20:10.844608+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:00:07.773428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195911,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2626,"num_read_kb":8389,"num_write":522,"num_write_kb":11747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"149'683","reported_seq":1687,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337358+0000","last_change":"2024-09-13T09:19:51.722579+0000","last_active":"2024-09-13T09:20:12.337358+0000","last_peered":"2024-09-13T09:20:12.337358+0000","last_clean":"2024-09-13T09:20:12.337358+0000","last_became_active":"2024-09-13T09:19:49.736505+0000","last_became_peered":"2024-09-13T09:19:49.736505+0000","last_unstale":"2024-09-13T09:20:12.337358+0000","last_undegraded":"2024-09-13T09:20:12.337358+0000","last_fullsized":"2024-09-13T09:20:12.337358+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:51:30.617730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00218943,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2689,"num_read_kb":8298,"num_write":946,"num_write_kb":9158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":43,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"147'693","reported_seq":3232,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338076+0000","last_change":"2024-09-13T09:19:51.724802+0000","last_active":"2024-09-13T09:20:12.338076+0000","last_peered":"2024-09-13T09:20:12.338076+0000","last_clean":"2024-09-13T09:20:12.338076+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:20:12.338076+0000","last_undegraded":"2024-09-13T09:20:12.338076+0000","last_fullsized":"2024-09-13T09:20:12.338076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:25.411265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035729350000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4417,"num_read_kb":31654,"num_write":734,"num_write_kb":30325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"146'383","reported_seq":1377,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338412+0000","last_change":"2024-09-13T09:19:51.721061+0000","last_active":"2024-09-13T09:20:12.338412+0000","last_peered":"2024-09-13T09:20:12.338412+0000","last_clean":"2024-09-13T09:20:12.338412+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:20:12.338412+0000","last_undegraded":"2024-09-13T09:20:12.338412+0000","last_fullsized":"2024-09-13T09:20:12.338412+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:31.868704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068946100000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1301,"num_read_kb":10607,"num_write":406,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"148'980","reported_seq":4053,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.338100+0000","last_change":"2024-09-13T09:19:51.722971+0000","last_active":"2024-09-13T09:20:12.338100+0000","last_peered":"2024-09-13T09:20:12.338100+0000","last_clean":"2024-09-13T09:20:12.338100+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:20:12.338100+0000","last_undegraded":"2024-09-13T09:20:12.338100+0000","last_fullsized":"2024-09-13T09:20:12.338100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:56.423075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024735249999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7129,"num_read_kb":22784,"num_write":1446,"num_write_kb":10311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"146'402","reported_seq":2308,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-13T09:20:08.902107+0000","last_change":"2024-09-13T09:19:51.826873+0000","last_active":"2024-09-13T09:20:08.902107+0000","last_peered":"2024-09-13T09:20:08.902107+0000","last_clean":"2024-09-13T09:20:08.902107+0000","last_became_active":"2024-09-13T09:19:20.803495+0000","last_became_peered":"2024-09-13T09:19:20.803495+0000","last_unstale":"2024-09-13T09:20:08.902107+0000","last_undegraded":"2024-09-13T09:20:08.902107+0000","last_fullsized":"2024-09-13T09:20:08.902107+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:18.288761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072899800000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2117,"num_read_kb":10151,"num_write":496,"num_write_kb":12845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5102507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"148'549","reported_seq":3010,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.336735+0000","last_change":"2024-09-13T09:19:51.722468+0000","last_active":"2024-09-13T09:20:12.336735+0000","last_peered":"2024-09-13T09:20:12.336735+0000","last_clean":"2024-09-13T09:20:12.336735+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:20:12.336735+0000","last_undegraded":"2024-09-13T09:20:12.336735+0000","last_fullsized":"2024-09-13T09:20:12.336735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:47.572839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020210359999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":14560,"num_write":713,"num_write_kb":15391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"146'438","reported_seq":4936,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:11.862012+0000","last_change":"2024-09-13T09:19:51.724223+0000","last_active":"2024-09-13T09:20:11.862012+0000","last_peered":"2024-09-13T09:20:11.862012+0000","last_clean":"2024-09-13T09:20:11.862012+0000","last_became_active":"2024-09-13T09:17:40.789499+0000","last_became_peered":"2024-09-13T09:17:40.789499+0000","last_unstale":"2024-09-13T09:20:11.862012+0000","last_undegraded":"2024-09-13T09:20:11.862012+0000","last_fullsized":"2024-09-13T09:20:11.862012+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010644070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5803,"num_read_kb":44996,"num_write":398,"num_write_kb":17054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"147'421","reported_seq":3620,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.337093+0000","last_change":"2024-09-13T09:19:51.723691+0000","last_active":"2024-09-13T09:20:12.337093+0000","last_peered":"2024-09-13T09:20:12.337093+0000","last_clean":"2024-09-13T09:20:12.337093+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:20:12.337093+0000","last_undegraded":"2024-09-13T09:20:12.337093+0000","last_fullsized":"2024-09-13T09:20:12.337093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:07.893848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003077476,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4122,"num_read_kb":18425,"num_write":408,"num_write_kb":9869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":152,"num_read":126915,"num_read_kb":555724,"num_write":19133,"num_write_kb":379149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1469,"num_bytes_recovered":28693609,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16684,"ondisk_log_size":16684,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":30095,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59380,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312460,"statfs":{"total":96636764160,"available":96575959040,"internally_reserved":0,"allocated":2068480,"data_stored":1548192,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.95699999999999996,"15min":0.89200000000000002},"min":{"1min":0.63500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.1799999999999999,"5min":11.683,"15min":11.683},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.0269999999999999,"15min":0.98099999999999998},"min":{"1min":0.60699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2729999999999999,"5min":11.843999999999999,"15min":11.843999999999999},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.85699999999999998,"15min":0.86299999999999999},"min":{"1min":0.54900000000000004,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.175,"5min":1.286,"15min":1.286},"last":1},{"interface":"front","average":{"1min":0.995,"5min":0.95099999999999996,"15min":0.98299999999999998},"min":{"1min":0.61699999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.304,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.82099999999999995,"15min":0.81000000000000005},"min":{"1min":0.53800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.042},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.73199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.0289999999999999,"15min":0.97099999999999997},"min":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.597,"5min":12.012,"15min":12.012},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.139,"15min":1.077},"min":{"1min":0.66400000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.615,"5min":11.776999999999999,"15min":11.776999999999999},"last":0.91300000000000003}]},{"osd":4,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.0049999999999999,"15min":1.0529999999999999},"min":{"1min":0.55700000000000005,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4319999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95799999999999996,"15min":0.96899999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.94599999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.079,"15min":1.073},"min":{"1min":0.59799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6990000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1160000000000001,"15min":1.143},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.651,"5min":3.125,"15min":3.125},"last":1.3360000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.163,"15min":1.222},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.9910000000000001,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":1.387},{"interface":"front","average":{"1min":1.163,"5min":1.117,"15min":1.1220000000000001},"min":{"1min":0.52000000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4750000000000001,"5min":1.996,"15min":1.996},"last":1.212}]}]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41272,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330568,"statfs":{"total":96636764160,"available":96594501632,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.438,"5min":0.314,"15min":0.314},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.75},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.877},"min":{"1min":0.40899999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3580000000000001,"5min":2.222,"15min":2.222},"last":1.109}]},{"osd":1,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.83399999999999996,"15min":0.81200000000000006},"min":{"1min":0.50800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.04,"5min":0.98199999999999998,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.607,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.232}]},{"osd":2,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.65700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4319999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.98499999999999999,"15min":1.0029999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.613,"5min":2.508,"15min":2.508},"last":1.1970000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.66900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.55,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.2450000000000001,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":1.1519999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.95199999999999996,"15min":0.97399999999999998},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2669999999999999,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.008,"15min":1},"min":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5800000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.0649999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.92300000000000004,"15min":0.90000000000000002},"min":{"1min":0.58999999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.57599999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.8120000000000001,"5min":2.286,"15min":2.286},"last":1.347}]},{"osd":7,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.093,"15min":1.087},"min":{"1min":0.621,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.611,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.079,"15min":1.0489999999999999},"min":{"1min":0.69899999999999995,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.274}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55236,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316604,"statfs":{"total":96636764160,"available":96580202496,"internally_reserved":0,"allocated":2015232,"data_stored":957843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":4.992,"15min":6.8739999999999997},"min":{"1min":0.47299999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.306,"5min":132.32900000000001,"15min":132.32900000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81499999999999995,"15min":0.81699999999999995},"min":{"1min":0.50900000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.151,"5min":1.335,"15min":1.335},"last":0.72699999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.84099999999999997,"15min":0.877},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2010000000000001,"5min":1.946,"15min":1.946},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97899999999999998,"15min":1.0149999999999999},"min":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4099999999999999,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84199999999999997,"15min":0.84399999999999997},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.875,"15min":0.89300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2130000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.80600000000000005}]},{"osd":3,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.94399999999999995},"min":{"1min":0.61299999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.286,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.98399999999999999,"15min":1},"min":{"1min":0.60899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.476,"15min":1.476},"last":0.64100000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.88900000000000001,"15min":0.872},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.458,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.103,"5min":5.375,"15min":7.2999999999999998},"min":{"1min":0.55100000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.395,"5min":132.23599999999999,"15min":132.23599999999999},"last":0.94099999999999995}]},{"osd":6,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.97099999999999997,"15min":0.96699999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.997,"5min":1.0069999999999999,"15min":1.03},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.5820000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.92400000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.085,"15min":1.0780000000000001},"min":{"1min":0.67900000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.359,"5min":1.655,"15min":1.655},"last":0.999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.115,"15min":1.131},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.4850000000000001,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":0.85799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51136,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320704,"statfs":{"total":96636764160,"available":96584400896,"internally_reserved":0,"allocated":2015232,"data_stored":957843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3560000000000001,"5min":1.357,"15min":1.357},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.94599999999999995,"15min":0.94799999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91400000000000003,"15min":0.92600000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.81100000000000005,"15min":0.81999999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.61499999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.877},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.006,"5min":0.88,"15min":0.84299999999999997},"min":{"1min":0.56999999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.68700000000000006}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86899999999999999,"15min":0.85399999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3300000000000001,"5min":1.575,"15min":1.575},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.069,"5min":1.0249999999999999,"15min":0.97799999999999998},"min":{"1min":0.628,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0899999999999999,"5min":4.7389999999999999,"15min":4.7389999999999999},"last":0.90600000000000003}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.45400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.97099999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0329999999999999},"min":{"1min":0.53500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.077,"15min":1.0629999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.4119999999999999,"5min":2.4119999999999999,"15min":2.4119999999999999},"last":1.175}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.034,"15min":1.026},"min":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5609999999999999,"5min":1.597,"15min":1.597},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.125,"15min":1.131},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.2030000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77372,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94294468,"statfs":{"total":96636764160,"available":96557535232,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.249,"5min":0.215,"15min":0.215},"max":{"1min":4.492,"5min":4.492,"15min":4.492},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.33100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.337,"5min":1.625,"15min":1.625},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.38600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.4990000000000001,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.72099999999999997,"15min":0.68899999999999995},"min":{"1min":0.29299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.3890000000000002,"5min":4.3890000000000002,"15min":4.3890000000000002},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.73099999999999998,"15min":0.70299999999999996},"min":{"1min":0.44,"5min":0.312,"15min":0.312},"max":{"1min":1.45,"5min":1.544,"15min":1.544},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.79600000000000004,"15min":0.77600000000000002},"min":{"1min":0.41699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.2880000000000003,"5min":4.2880000000000003,"15min":4.2880000000000003},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.90400000000000003,"15min":0.91000000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.94899999999999995,"15min":0.94699999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.0529999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0049999999999999,"15min":0.98299999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":5.3319999999999999,"5min":5.3319999999999999,"15min":5.3319999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.93899999999999995,"15min":0.93700000000000006},"min":{"1min":0.55200000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.96999999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.022,"15min":1.0289999999999999},"min":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.1200000000000001,"15min":1.1080000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":5.25,"5min":5.25,"15min":5.25},"last":1.1870000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.0149999999999999,"15min":1.012},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.6899999999999999,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.5269999999999999,"15min":1.726},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.504},"max":{"1min":2.097,"5min":15.4,"15min":15.4},"last":1.2170000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41272,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330568,"statfs":{"total":96636764160,"available":96594501632,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.621},"min":{"1min":0.247,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1899999999999999,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.72499999999999998,"15min":0.70499999999999996},"min":{"1min":0.28399999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.671,"5min":2.8490000000000002,"15min":2.8490000000000002},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.79000000000000004,"15min":0.79300000000000004},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.337,"5min":1.907,"15min":1.907},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.77600000000000002,"15min":0.74399999999999999},"min":{"1min":0.48999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.256,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.83899999999999997,"15min":0.871},"min":{"1min":0.34000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.47,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.88400000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.97399999999999998,"15min":0.96999999999999997},"min":{"1min":0.76200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4470000000000001,"5min":1.782,"15min":1.782},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.83499999999999996,"15min":0.81699999999999995},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.96199999999999997}]},{"osd":5,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.97999999999999998,"15min":0.96799999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.212},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0329999999999999,"15min":1.0289999999999999},"min":{"1min":0.57199999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5580000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2490000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.55600000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.012,"15min":1.0049999999999999},"min":{"1min":0.623,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.923,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.1220000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.121,"15min":1.133},"min":{"1min":0.68899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.573,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.099},{"interface":"front","average":{"1min":1.071,"5min":1.0049999999999999,"15min":1.0029999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5620000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.0089999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":2019328,"data_stored":957862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.623},"min":{"1min":0.35799999999999998,"5min":0.255,"15min":0.255},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.34300000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.95499999999999996,"5min":1.484,"15min":1.484},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72899999999999998,"15min":0.75700000000000001},"min":{"1min":0.36699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.0569999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.753},"min":{"1min":0.44600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.135,"5min":1.264,"15min":1.264},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76500000000000001,"15min":0.76900000000000002},"min":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1499999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.495}]},{"osd":4,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3080000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97699999999999998,"15min":1.0109999999999999},"min":{"1min":0.64800000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3620000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.76200000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0780000000000001,"15min":1.0409999999999999},"min":{"1min":0.60799999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.524,"5min":8.3460000000000001,"15min":8.3460000000000001},"last":0.871},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98799999999999999,"15min":1.018},"min":{"1min":0.56599999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4710000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.68100000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.01,"15min":1.044},"min":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.573,"5min":1.647,"15min":1.647},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.002,"15min":1.028},"min":{"1min":0.61499999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.623}]},{"osd":7,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.024,"15min":1.05},"min":{"1min":0.69099999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5800000000000001,"5min":1.629,"15min":1.629},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.081,"15min":1.1140000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.60999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":2052096,"data_stored":1548192,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.316,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.325,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.625,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.006,"5min":1.284,"15min":1.284},"last":0.751}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92600000000000005,"5min":1.5,"15min":1.5},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.39000000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.129,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.7989999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.72499999999999998,"15min":0.69899999999999995},"min":{"1min":0.42699999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.452,"5min":2.242,"15min":2.242},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.78900000000000003},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2250000000000001,"5min":1.528,"15min":1.528},"last":0.80700000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.89200000000000002,"15min":0.877},"min":{"1min":0.53700000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.8660000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0309999999999999,"15min":0.96599999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.484,"5min":9.5649999999999995,"15min":9.5649999999999995},"last":0.78600000000000003}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0469999999999999,"15min":0.95799999999999996},"min":{"1min":0.67900000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.375,"5min":9.6120000000000001,"15min":9.6120000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.419,"5min":1.476,"15min":1.476},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0620000000000001,"15min":1.085},"min":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4279999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.996,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.482,"5min":1.516,"15min":1.516},"last":1.8360000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0229999999999999,"15min":1.0309999999999999},"min":{"1min":0.58799999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.5669999999999999,"5min":1.617,"15min":1.617},"last":1.8919999999999999},{"interface":"front","average":{"1min":0.995,"5min":1.1379999999999999,"15min":1.0820000000000001},"min":{"1min":0.67300000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.546,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":0.84799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:20:22.399 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.399 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.399 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.399 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.400 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.400 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.400 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.400 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.400 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.400 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.400 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.400 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.400 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-13T09:17:16.469202+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=16, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.401 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.401 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T09:17:16.469202+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=16, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.401 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.401 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T09:17:16.469202+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=16, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.401 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.401 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T09:17:16.469202+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=16, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.401 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.401 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.402 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T09:17:16.469202+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=16, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.402 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.402 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-13T09:17:16.469202+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=16, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.402 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.402 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T09:17:16.469202+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=16, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.402 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.402 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T09:17:16.469202+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=16, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.402 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T09:17:10.373878+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=10, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.402 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.403 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.403 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.403 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.403 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.403 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.403 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.403 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.403 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.403 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.403 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.404 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:22.404 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T09:20:42.404 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:20:42.729 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:20:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:20:42.747 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-13T09:20:42.114221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":152,"num_read":127031,"num_read_kb":555824,"num_write":19294,"num_write_kb":381567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1476,"num_bytes_recovered":29874402,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16943,"ondisk_log_size":16943,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":391824,"kb_used_data":15824,"kb_used_omap":12,"kb_used_meta":375923,"kb_avail":754582896,"statfs":{"total":773094113280,"available":772692885504,"internally_reserved":0,"allocated":16203776,"data_stored":8843404,"data_compressed":19717,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12744,"internal_metadata":384945720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003791"},"pg_stats":[{"pgid":"3.1f","version":"167'408","reported_seq":1897,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:30.988502+0000","last_change":"2024-09-13T09:20:30.988734+0000","last_active":"2024-09-13T09:20:30.988502+0000","last_peered":"2024-09-13T09:20:30.988502+0000","last_clean":"2024-09-13T09:20:30.988502+0000","last_became_active":"2024-09-13T09:17:40.791187+0000","last_became_peered":"2024-09-13T09:17:40.791187+0000","last_unstale":"2024-09-13T09:20:30.988502+0000","last_undegraded":"2024-09-13T09:20:30.988502+0000","last_fullsized":"2024-09-13T09:20:30.988502+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'408","last_scrub_stamp":"2024-09-13T09:20:30.988449+0000","last_deep_scrub":"167'408","last_deep_scrub_stamp":"2024-09-13T09:20:30.988449+0000","last_clean_scrub_stamp":"2024-09-13T09:20:30.988449+0000","objects_scrubbed":2,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:28:22.162575+0000","scrub_duration":0.0078384240000000001,"objects_trimmed":0,"snaptrim_duration":0.001951139,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2539,"num_read_kb":13964,"num_write":479,"num_write_kb":12323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"167'646","reported_seq":3301,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:22.201025+0000","last_change":"2024-09-13T09:20:22.201131+0000","last_active":"2024-09-13T09:20:22.201025+0000","last_peered":"2024-09-13T09:20:22.201025+0000","last_clean":"2024-09-13T09:20:22.201025+0000","last_became_active":"2024-09-13T09:19:20.802379+0000","last_became_peered":"2024-09-13T09:19:20.802379+0000","last_unstale":"2024-09-13T09:20:22.201025+0000","last_undegraded":"2024-09-13T09:20:22.201025+0000","last_fullsized":"2024-09-13T09:20:22.201025+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'646","last_scrub_stamp":"2024-09-13T09:20:22.201003+0000","last_deep_scrub":"167'646","last_deep_scrub_stamp":"2024-09-13T09:20:22.201003+0000","last_clean_scrub_stamp":"2024-09-13T09:20:22.201003+0000","objects_scrubbed":6,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:35.776557+0000","scrub_duration":0.010201534,"objects_trimmed":0,"snaptrim_duration":0.001400552,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3630,"num_read_kb":10038,"num_write":856,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":8249616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"167'493","reported_seq":3779,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:37.041436+0000","last_change":"2024-09-13T09:20:37.041704+0000","last_active":"2024-09-13T09:20:37.041436+0000","last_peered":"2024-09-13T09:20:37.041436+0000","last_clean":"2024-09-13T09:20:37.041436+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:20:37.041436+0000","last_undegraded":"2024-09-13T09:20:37.041436+0000","last_fullsized":"2024-09-13T09:20:37.041436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'493","last_scrub_stamp":"2024-09-13T09:20:37.041380+0000","last_deep_scrub":"167'493","last_deep_scrub_stamp":"2024-09-13T09:20:37.041380+0000","last_clean_scrub_stamp":"2024-09-13T09:20:37.041380+0000","objects_scrubbed":5,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:34.537298+0000","scrub_duration":0.012442204,"objects_trimmed":0,"snaptrim_duration":0.0021011329999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5768,"num_read_kb":20052,"num_write":439,"num_write_kb":19543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"167'796","reported_seq":4580,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:36.013488+0000","last_change":"2024-09-13T09:20:36.013737+0000","last_active":"2024-09-13T09:20:36.013488+0000","last_peered":"2024-09-13T09:20:36.013488+0000","last_clean":"2024-09-13T09:20:36.013488+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:20:36.013488+0000","last_undegraded":"2024-09-13T09:20:36.013488+0000","last_fullsized":"2024-09-13T09:20:36.013488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'796","last_scrub_stamp":"2024-09-13T09:20:36.013433+0000","last_deep_scrub":"167'796","last_deep_scrub_stamp":"2024-09-13T09:20:36.013433+0000","last_clean_scrub_stamp":"2024-09-13T09:20:36.013433+0000","objects_scrubbed":3,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:44.935327+0000","scrub_duration":0.010464815000000001,"objects_trimmed":0,"snaptrim_duration":0.0025684319999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":7779,"num_read_kb":18614,"num_write":881,"num_write_kb":10036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"167'533","reported_seq":4445,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:26.898210+0000","last_change":"2024-09-13T09:20:26.898292+0000","last_active":"2024-09-13T09:20:26.898210+0000","last_peered":"2024-09-13T09:20:26.898210+0000","last_clean":"2024-09-13T09:20:26.898210+0000","last_became_active":"2024-09-13T09:17:40.799618+0000","last_became_peered":"2024-09-13T09:17:40.799618+0000","last_unstale":"2024-09-13T09:20:26.898210+0000","last_undegraded":"2024-09-13T09:20:26.898210+0000","last_fullsized":"2024-09-13T09:20:26.898210+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'533","last_scrub_stamp":"2024-09-13T09:20:26.898189+0000","last_deep_scrub":"167'533","last_deep_scrub_stamp":"2024-09-13T09:20:26.898189+0000","last_clean_scrub_stamp":"2024-09-13T09:20:26.898189+0000","objects_scrubbed":2,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:18:27.402049+0000","scrub_duration":0.0061538280000000001,"objects_trimmed":0,"snaptrim_duration":0.001823089,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5766,"num_read_kb":19616,"num_write":659,"num_write_kb":9141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":65341,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"167'352","reported_seq":3308,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:26.894646+0000","last_change":"2024-09-13T09:20:26.894914+0000","last_active":"2024-09-13T09:20:26.894646+0000","last_peered":"2024-09-13T09:20:26.894646+0000","last_clean":"2024-09-13T09:20:26.894646+0000","last_became_active":"2024-09-13T09:18:03.973063+0000","last_became_peered":"2024-09-13T09:18:03.973063+0000","last_unstale":"2024-09-13T09:20:26.894646+0000","last_undegraded":"2024-09-13T09:20:26.894646+0000","last_fullsized":"2024-09-13T09:20:26.894646+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'352","last_scrub_stamp":"2024-09-13T09:20:26.894594+0000","last_deep_scrub":"167'352","last_deep_scrub_stamp":"2024-09-13T09:20:26.894594+0000","last_clean_scrub_stamp":"2024-09-13T09:20:26.894594+0000","objects_scrubbed":8,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:58.666673+0000","scrub_duration":0.013708579,"objects_trimmed":0,"snaptrim_duration":0.00082503800000000005,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3197,"num_read_kb":8961,"num_write":344,"num_write_kb":6710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":889520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"167'725","reported_seq":4273,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:35.813651+0000","last_change":"2024-09-13T09:20:35.813923+0000","last_active":"2024-09-13T09:20:35.813651+0000","last_peered":"2024-09-13T09:20:35.813651+0000","last_clean":"2024-09-13T09:20:35.813651+0000","last_became_active":"2024-09-13T09:18:06.975016+0000","last_became_peered":"2024-09-13T09:18:06.975016+0000","last_unstale":"2024-09-13T09:20:35.813651+0000","last_undegraded":"2024-09-13T09:20:35.813651+0000","last_fullsized":"2024-09-13T09:20:35.813651+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'725","last_scrub_stamp":"2024-09-13T09:20:35.813596+0000","last_deep_scrub":"167'725","last_deep_scrub_stamp":"2024-09-13T09:20:35.813596+0000","last_clean_scrub_stamp":"2024-09-13T09:20:35.813596+0000","objects_scrubbed":10,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:48.519519+0000","scrub_duration":0.021453799999999999,"objects_trimmed":0,"snaptrim_duration":0.0010502549999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5493,"num_read_kb":28565,"num_write":1011,"num_write_kb":11903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":439303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"167'223","reported_seq":2579,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:36.085607+0000","last_change":"2024-09-13T09:20:36.085832+0000","last_active":"2024-09-13T09:20:36.085607+0000","last_peered":"2024-09-13T09:20:36.085607+0000","last_clean":"2024-09-13T09:20:36.085607+0000","last_became_active":"2024-09-13T09:17:40.797607+0000","last_became_peered":"2024-09-13T09:17:40.797607+0000","last_unstale":"2024-09-13T09:20:36.085607+0000","last_undegraded":"2024-09-13T09:20:36.085607+0000","last_fullsized":"2024-09-13T09:20:36.085607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'223","last_scrub_stamp":"2024-09-13T09:20:36.085553+0000","last_deep_scrub":"167'223","last_deep_scrub_stamp":"2024-09-13T09:20:36.085553+0000","last_clean_scrub_stamp":"2024-09-13T09:20:36.085553+0000","objects_scrubbed":6,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:44:53.450646+0000","scrub_duration":0.012966777000000001,"objects_trimmed":0,"snaptrim_duration":0.0011389530000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2065,"num_read_kb":17454,"num_write":181,"num_write_kb":10975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":42856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"167'445","reported_seq":2316,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:19.859558+0000","last_change":"2024-09-13T09:20:19.859660+0000","last_active":"2024-09-13T09:20:19.859558+0000","last_peered":"2024-09-13T09:20:19.859558+0000","last_clean":"2024-09-13T09:20:19.859558+0000","last_became_active":"2024-09-13T09:17:43.809533+0000","last_became_peered":"2024-09-13T09:17:43.809533+0000","last_unstale":"2024-09-13T09:20:19.859558+0000","last_undegraded":"2024-09-13T09:20:19.859558+0000","last_fullsized":"2024-09-13T09:20:19.859558+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'445","last_scrub_stamp":"2024-09-13T09:20:19.859539+0000","last_deep_scrub":"167'445","last_deep_scrub_stamp":"2024-09-13T09:20:19.859539+0000","last_clean_scrub_stamp":"2024-09-13T09:20:19.859539+0000","objects_scrubbed":3,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:08:48.625415+0000","scrub_duration":0.0070617070000000004,"objects_trimmed":0,"snaptrim_duration":0.0018671040000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3080,"num_read_kb":24506,"num_write":379,"num_write_kb":12955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":16384,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"167'278","reported_seq":2302,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:35.003177+0000","last_change":"2024-09-13T09:20:35.003481+0000","last_active":"2024-09-13T09:20:35.003177+0000","last_peered":"2024-09-13T09:20:35.003177+0000","last_clean":"2024-09-13T09:20:35.003177+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:20:35.003177+0000","last_undegraded":"2024-09-13T09:20:35.003177+0000","last_fullsized":"2024-09-13T09:20:35.003177+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'278","last_scrub_stamp":"2024-09-13T09:20:35.003118+0000","last_deep_scrub":"167'278","last_deep_scrub_stamp":"2024-09-13T09:20:35.003118+0000","last_clean_scrub_stamp":"2024-09-13T09:20:35.003118+0000","objects_scrubbed":6,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:55:51.435272+0000","scrub_duration":0.013897542000000001,"objects_trimmed":0,"snaptrim_duration":0.001510742,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1901,"num_read_kb":15086,"num_write":247,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"147'371","reported_seq":2969,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.727328+0000","last_change":"2024-09-13T09:19:51.723974+0000","last_active":"2024-09-13T09:20:12.727328+0000","last_peered":"2024-09-13T09:20:12.727328+0000","last_clean":"2024-09-13T09:20:12.727328+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:20:12.727328+0000","last_undegraded":"2024-09-13T09:20:12.727328+0000","last_fullsized":"2024-09-13T09:20:12.727328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029658509999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3481,"num_read_kb":18288,"num_write":412,"num_write_kb":15933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"167'584","reported_seq":3095,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:29.876176+0000","last_change":"2024-09-13T09:20:29.876393+0000","last_active":"2024-09-13T09:20:29.876176+0000","last_peered":"2024-09-13T09:20:29.876176+0000","last_clean":"2024-09-13T09:20:29.876176+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:20:29.876176+0000","last_undegraded":"2024-09-13T09:20:29.876176+0000","last_fullsized":"2024-09-13T09:20:29.876176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'584","last_scrub_stamp":"2024-09-13T09:20:29.876121+0000","last_deep_scrub":"167'584","last_deep_scrub_stamp":"2024-09-13T09:20:29.876121+0000","last_clean_scrub_stamp":"2024-09-13T09:20:29.876121+0000","objects_scrubbed":4,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:49.387878+0000","scrub_duration":0.011138855,"objects_trimmed":0,"snaptrim_duration":0.001048134,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3471,"num_read_kb":17629,"num_write":764,"num_write_kb":18883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:24.305923+0000","last_change":"2024-09-13T09:20:24.306194+0000","last_active":"2024-09-13T09:20:24.305923+0000","last_peered":"2024-09-13T09:20:24.305923+0000","last_clean":"2024-09-13T09:20:24.305923+0000","last_became_active":"2024-09-13T09:19:17.796177+0000","last_became_peered":"2024-09-13T09:19:17.796177+0000","last_unstale":"2024-09-13T09:20:24.305923+0000","last_undegraded":"2024-09-13T09:20:24.305923+0000","last_fullsized":"2024-09-13T09:20:24.305923+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:20:24.305847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:20:24.305847+0000","last_clean_scrub_stamp":"2024-09-13T09:20:24.305847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:51:09.170716+0000","scrub_duration":0.0030267710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"167'454","reported_seq":1845,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:32.020794+0000","last_change":"2024-09-13T09:20:32.021125+0000","last_active":"2024-09-13T09:20:32.020794+0000","last_peered":"2024-09-13T09:20:32.020794+0000","last_clean":"2024-09-13T09:20:32.020794+0000","last_became_active":"2024-09-13T09:19:14.748293+0000","last_became_peered":"2024-09-13T09:19:14.748293+0000","last_unstale":"2024-09-13T09:20:32.020794+0000","last_undegraded":"2024-09-13T09:20:32.020794+0000","last_fullsized":"2024-09-13T09:20:32.020794+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'454","last_scrub_stamp":"2024-09-13T09:20:32.020683+0000","last_deep_scrub":"167'454","last_deep_scrub_stamp":"2024-09-13T09:20:32.020683+0000","last_clean_scrub_stamp":"2024-09-13T09:20:32.020683+0000","objects_scrubbed":4,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:58:59.331014+0000","scrub_duration":0.010434268999999999,"objects_trimmed":0,"snaptrim_duration":0.0016656990000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2578,"num_read_kb":17393,"num_write":357,"num_write_kb":10507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":7471802,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":157,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:28.771225+0000","last_change":"2024-09-13T09:20:28.771488+0000","last_active":"2024-09-13T09:20:28.771225+0000","last_peered":"2024-09-13T09:20:28.771225+0000","last_clean":"2024-09-13T09:20:28.771225+0000","last_became_active":"2024-09-13T09:19:17.769570+0000","last_became_peered":"2024-09-13T09:19:17.769570+0000","last_unstale":"2024-09-13T09:20:28.771225+0000","last_undegraded":"2024-09-13T09:20:28.771225+0000","last_fullsized":"2024-09-13T09:20:28.771225+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:20:28.771166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:20:28.771166+0000","last_clean_scrub_stamp":"2024-09-13T09:20:28.771166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:43.434175+0000","scrub_duration":0.0032507159999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"167'565","reported_seq":3028,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:28.188435+0000","last_change":"2024-09-13T09:20:28.188667+0000","last_active":"2024-09-13T09:20:28.188435+0000","last_peered":"2024-09-13T09:20:28.188435+0000","last_clean":"2024-09-13T09:20:28.188435+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:20:28.188435+0000","last_undegraded":"2024-09-13T09:20:28.188435+0000","last_fullsized":"2024-09-13T09:20:28.188435+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'565","last_scrub_stamp":"2024-09-13T09:20:28.188378+0000","last_deep_scrub":"167'565","last_deep_scrub_stamp":"2024-09-13T09:20:28.188378+0000","last_clean_scrub_stamp":"2024-09-13T09:20:28.188378+0000","objects_scrubbed":4,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:13.700620+0000","scrub_duration":0.010427054999999999,"objects_trimmed":0,"snaptrim_duration":0.0015869739999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4402,"num_read_kb":8559,"num_write":727,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":189,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:27.775438+0000","last_change":"2024-09-13T09:20:27.775674+0000","last_active":"2024-09-13T09:20:27.775438+0000","last_peered":"2024-09-13T09:20:27.775438+0000","last_clean":"2024-09-13T09:20:27.775438+0000","last_became_active":"2024-09-13T09:17:40.793882+0000","last_became_peered":"2024-09-13T09:17:40.793882+0000","last_unstale":"2024-09-13T09:20:27.775438+0000","last_undegraded":"2024-09-13T09:20:27.775438+0000","last_fullsized":"2024-09-13T09:20:27.775438+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:20:27.775385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:20:27.775385+0000","last_clean_scrub_stamp":"2024-09-13T09:20:27.775385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:01.437595+0000","scrub_duration":0.003321231,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"167'385","reported_seq":1642,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:31.759985+0000","last_change":"2024-09-13T09:20:31.760219+0000","last_active":"2024-09-13T09:20:31.759985+0000","last_peered":"2024-09-13T09:20:31.759985+0000","last_clean":"2024-09-13T09:20:31.759985+0000","last_became_active":"2024-09-13T09:18:03.973314+0000","last_became_peered":"2024-09-13T09:18:03.973314+0000","last_unstale":"2024-09-13T09:20:31.759985+0000","last_undegraded":"2024-09-13T09:20:31.759985+0000","last_fullsized":"2024-09-13T09:20:31.759985+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'385","last_scrub_stamp":"2024-09-13T09:20:31.759894+0000","last_deep_scrub":"167'385","last_deep_scrub_stamp":"2024-09-13T09:20:31.759894+0000","last_clean_scrub_stamp":"2024-09-13T09:20:31.759894+0000","objects_scrubbed":8,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:31:47.761295+0000","scrub_duration":0.015260414999999999,"objects_trimmed":0,"snaptrim_duration":0.0012376119999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1399,"num_read_kb":26203,"num_write":287,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":167476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:34.773034+0000","last_change":"2024-09-13T09:20:34.773326+0000","last_active":"2024-09-13T09:20:34.773034+0000","last_peered":"2024-09-13T09:20:34.773034+0000","last_clean":"2024-09-13T09:20:34.773034+0000","last_became_active":"2024-09-13T09:18:03.967133+0000","last_became_peered":"2024-09-13T09:18:03.967133+0000","last_unstale":"2024-09-13T09:20:34.773034+0000","last_undegraded":"2024-09-13T09:20:34.773034+0000","last_fullsized":"2024-09-13T09:20:34.773034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:20:34.772980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:20:34.772980+0000","last_clean_scrub_stamp":"2024-09-13T09:20:34.772980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:18:21.081636+0000","scrub_duration":0.0031266240000000002,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"167'349","reported_seq":2285,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:34.010284+0000","last_change":"2024-09-13T09:20:34.010523+0000","last_active":"2024-09-13T09:20:34.010284+0000","last_peered":"2024-09-13T09:20:34.010284+0000","last_clean":"2024-09-13T09:20:34.010284+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:20:34.010284+0000","last_undegraded":"2024-09-13T09:20:34.010284+0000","last_fullsized":"2024-09-13T09:20:34.010284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'349","last_scrub_stamp":"2024-09-13T09:20:34.010225+0000","last_deep_scrub":"167'349","last_deep_scrub_stamp":"2024-09-13T09:20:34.010225+0000","last_clean_scrub_stamp":"2024-09-13T09:20:34.010225+0000","objects_scrubbed":5,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:59:37.255952+0000","scrub_duration":0.01062372,"objects_trimmed":0,"snaptrim_duration":0.00109892,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2142,"num_read_kb":9020,"num_write":356,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"167'659","reported_seq":4767,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:25.868457+0000","last_change":"2024-09-13T09:20:25.868691+0000","last_active":"2024-09-13T09:20:25.868457+0000","last_peered":"2024-09-13T09:20:25.868457+0000","last_clean":"2024-09-13T09:20:25.868457+0000","last_became_active":"2024-09-13T09:17:40.800027+0000","last_became_peered":"2024-09-13T09:17:40.800027+0000","last_unstale":"2024-09-13T09:20:25.868457+0000","last_undegraded":"2024-09-13T09:20:25.868457+0000","last_fullsized":"2024-09-13T09:20:25.868457+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'659","last_scrub_stamp":"2024-09-13T09:20:25.868397+0000","last_deep_scrub":"167'659","last_deep_scrub_stamp":"2024-09-13T09:20:25.868397+0000","last_clean_scrub_stamp":"2024-09-13T09:20:25.868397+0000","objects_scrubbed":4,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:14.674500+0000","scrub_duration":0.010044911,"objects_trimmed":0,"snaptrim_duration":0.00072429099999999995,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":5951,"num_read_kb":14391,"num_write":711,"num_write_kb":6974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":29248,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"167'3","reported_seq":212,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:25.937343+0000","last_change":"2024-09-13T09:20:25.937597+0000","last_active":"2024-09-13T09:20:25.937343+0000","last_peered":"2024-09-13T09:20:25.937343+0000","last_clean":"2024-09-13T09:20:25.937343+0000","last_became_active":"2024-09-13T09:18:03.970835+0000","last_became_peered":"2024-09-13T09:18:03.970835+0000","last_unstale":"2024-09-13T09:20:25.937343+0000","last_undegraded":"2024-09-13T09:20:25.937343+0000","last_fullsized":"2024-09-13T09:20:25.937343+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'3","last_scrub_stamp":"2024-09-13T09:20:25.937291+0000","last_deep_scrub":"167'3","last_deep_scrub_stamp":"2024-09-13T09:20:25.937291+0000","last_clean_scrub_stamp":"2024-09-13T09:20:25.937291+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:43.100284+0000","scrub_duration":0.0060844330000000002,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"167'565","reported_seq":2696,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:33.758728+0000","last_change":"2024-09-13T09:20:33.758978+0000","last_active":"2024-09-13T09:20:33.758728+0000","last_peered":"2024-09-13T09:20:33.758728+0000","last_clean":"2024-09-13T09:20:33.758728+0000","last_became_active":"2024-09-13T09:18:03.970479+0000","last_became_peered":"2024-09-13T09:18:03.970479+0000","last_unstale":"2024-09-13T09:20:33.758728+0000","last_undegraded":"2024-09-13T09:20:33.758728+0000","last_fullsized":"2024-09-13T09:20:33.758728+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'565","last_scrub_stamp":"2024-09-13T09:20:33.758671+0000","last_deep_scrub":"167'565","last_deep_scrub_stamp":"2024-09-13T09:20:33.758671+0000","last_clean_scrub_stamp":"2024-09-13T09:20:33.758671+0000","objects_scrubbed":4,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:42:11.958274+0000","scrub_duration":0.010858072,"objects_trimmed":0,"snaptrim_duration":0.001031719,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5538,"num_read_kb":12570,"num_write":635,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1031390,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:21.445039+0000","last_change":"2024-09-13T09:20:21.445137+0000","last_active":"2024-09-13T09:20:21.445039+0000","last_peered":"2024-09-13T09:20:21.445039+0000","last_clean":"2024-09-13T09:20:21.445039+0000","last_became_active":"2024-09-13T09:20:02.756499+0000","last_became_peered":"2024-09-13T09:20:02.756499+0000","last_unstale":"2024-09-13T09:20:21.445039+0000","last_undegraded":"2024-09-13T09:20:21.445039+0000","last_fullsized":"2024-09-13T09:20:21.445039+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:20:21.445020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:20:21.445020+0000","last_clean_scrub_stamp":"2024-09-13T09:20:21.445020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:38:10.626621+0000","scrub_duration":0.0013369810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"167'490","reported_seq":3465,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:20.907209+0000","last_change":"2024-09-13T09:20:20.907319+0000","last_active":"2024-09-13T09:20:20.907209+0000","last_peered":"2024-09-13T09:20:20.907209+0000","last_clean":"2024-09-13T09:20:20.907209+0000","last_became_active":"2024-09-13T09:17:40.793065+0000","last_became_peered":"2024-09-13T09:17:40.793065+0000","last_unstale":"2024-09-13T09:20:20.907209+0000","last_undegraded":"2024-09-13T09:20:20.907209+0000","last_fullsized":"2024-09-13T09:20:20.907209+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'490","last_scrub_stamp":"2024-09-13T09:20:20.907181+0000","last_deep_scrub":"167'490","last_deep_scrub_stamp":"2024-09-13T09:20:20.907181+0000","last_clean_scrub_stamp":"2024-09-13T09:20:20.907181+0000","objects_scrubbed":7,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:09.277243+0000","scrub_duration":0.01046126,"objects_trimmed":0,"snaptrim_duration":0.00175362,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4025,"num_read_kb":26901,"num_write":443,"num_write_kb":18254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":190,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:24.778333+0000","last_change":"2024-09-13T09:20:24.778605+0000","last_active":"2024-09-13T09:20:24.778333+0000","last_peered":"2024-09-13T09:20:24.778333+0000","last_clean":"2024-09-13T09:20:24.778333+0000","last_became_active":"2024-09-13T09:18:03.968092+0000","last_became_peered":"2024-09-13T09:18:03.968092+0000","last_unstale":"2024-09-13T09:20:24.778333+0000","last_undegraded":"2024-09-13T09:20:24.778333+0000","last_fullsized":"2024-09-13T09:20:24.778333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:20:24.778284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:20:24.778284+0000","last_clean_scrub_stamp":"2024-09-13T09:20:24.778284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:36:17.198676+0000","scrub_duration":0.0033279809999999998,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"167'790","reported_seq":2807,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:27.243135+0000","last_change":"2024-09-13T09:20:27.243385+0000","last_active":"2024-09-13T09:20:27.243135+0000","last_peered":"2024-09-13T09:20:27.243135+0000","last_clean":"2024-09-13T09:20:27.243135+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:20:27.243135+0000","last_undegraded":"2024-09-13T09:20:27.243135+0000","last_fullsized":"2024-09-13T09:20:27.243135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'790","last_scrub_stamp":"2024-09-13T09:20:27.243075+0000","last_deep_scrub":"167'790","last_deep_scrub_stamp":"2024-09-13T09:20:27.243075+0000","last_clean_scrub_stamp":"2024-09-13T09:20:27.243075+0000","objects_scrubbed":8,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:54:21.889458+0000","scrub_duration":0.018141651000000002,"objects_trimmed":0,"snaptrim_duration":0.0012273519999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4598,"num_read_kb":13738,"num_write":1068,"num_write_kb":11812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"167'2","reported_seq":175,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:29.003759+0000","last_change":"2024-09-13T09:20:29.004066+0000","last_active":"2024-09-13T09:20:29.003759+0000","last_peered":"2024-09-13T09:20:29.003759+0000","last_clean":"2024-09-13T09:20:29.003759+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:20:29.003759+0000","last_undegraded":"2024-09-13T09:20:29.003759+0000","last_fullsized":"2024-09-13T09:20:29.003759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'2","last_scrub_stamp":"2024-09-13T09:20:29.003640+0000","last_deep_scrub":"167'2","last_deep_scrub_stamp":"2024-09-13T09:20:29.003640+0000","last_clean_scrub_stamp":"2024-09-13T09:20:29.003640+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:40:04.692771+0000","scrub_duration":0.0064562659999999996,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"167'119","reported_seq":366,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:22.796156+0000","last_change":"2024-09-13T09:20:22.796429+0000","last_active":"2024-09-13T09:20:22.796156+0000","last_peered":"2024-09-13T09:20:22.796156+0000","last_clean":"2024-09-13T09:20:22.796156+0000","last_became_active":"2024-09-13T09:17:43.802603+0000","last_became_peered":"2024-09-13T09:17:43.802603+0000","last_unstale":"2024-09-13T09:20:22.796156+0000","last_undegraded":"2024-09-13T09:20:22.796156+0000","last_fullsized":"2024-09-13T09:20:22.796156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'119","last_scrub_stamp":"2024-09-13T09:20:22.796093+0000","last_deep_scrub":"167'119","last_deep_scrub_stamp":"2024-09-13T09:20:22.796093+0000","last_clean_scrub_stamp":"2024-09-13T09:20:22.796093+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:11.050876+0000","scrub_duration":0.0089725240000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"167'657","reported_seq":2743,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:32.156565+0000","last_change":"2024-09-13T09:20:32.156845+0000","last_active":"2024-09-13T09:20:32.156565+0000","last_peered":"2024-09-13T09:20:32.156565+0000","last_clean":"2024-09-13T09:20:32.156565+0000","last_became_active":"2024-09-13T09:18:03.974569+0000","last_became_peered":"2024-09-13T09:18:03.974569+0000","last_unstale":"2024-09-13T09:20:32.156565+0000","last_undegraded":"2024-09-13T09:20:32.156565+0000","last_fullsized":"2024-09-13T09:20:32.156565+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'657","last_scrub_stamp":"2024-09-13T09:20:32.156509+0000","last_deep_scrub":"167'657","last_deep_scrub_stamp":"2024-09-13T09:20:32.156509+0000","last_clean_scrub_stamp":"2024-09-13T09:20:32.156509+0000","objects_scrubbed":7,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:15.684735+0000","scrub_duration":0.015877624,"objects_trimmed":0,"snaptrim_duration":0.0036429610000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4126,"num_read_kb":16863,"num_write":738,"num_write_kb":13412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":741496,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"167'543","reported_seq":2828,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:31.845860+0000","last_change":"2024-09-13T09:20:31.846112+0000","last_active":"2024-09-13T09:20:31.845860+0000","last_peered":"2024-09-13T09:20:31.845860+0000","last_clean":"2024-09-13T09:20:31.845860+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:20:31.845860+0000","last_undegraded":"2024-09-13T09:20:31.845860+0000","last_fullsized":"2024-09-13T09:20:31.845860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'543","last_scrub_stamp":"2024-09-13T09:20:31.845778+0000","last_deep_scrub":"167'543","last_deep_scrub_stamp":"2024-09-13T09:20:31.845778+0000","last_clean_scrub_stamp":"2024-09-13T09:20:31.845778+0000","objects_scrubbed":4,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:04.926730+0000","scrub_duration":0.010190695,"objects_trimmed":0,"snaptrim_duration":0.00070054600000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":11592,"num_write":623,"num_write_kb":5754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"167'509","reported_seq":3894,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:26.289700+0000","last_change":"2024-09-13T09:20:26.289939+0000","last_active":"2024-09-13T09:20:26.289700+0000","last_peered":"2024-09-13T09:20:26.289700+0000","last_clean":"2024-09-13T09:20:26.289700+0000","last_became_active":"2024-09-13T09:19:34.891064+0000","last_became_peered":"2024-09-13T09:19:34.891064+0000","last_unstale":"2024-09-13T09:20:26.289700+0000","last_undegraded":"2024-09-13T09:20:26.289700+0000","last_fullsized":"2024-09-13T09:20:26.289700+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'509","last_scrub_stamp":"2024-09-13T09:20:26.289647+0000","last_deep_scrub":"167'509","last_deep_scrub_stamp":"2024-09-13T09:20:26.289647+0000","last_clean_scrub_stamp":"2024-09-13T09:20:26.289647+0000","objects_scrubbed":4,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:06:11.262198+0000","scrub_duration":0.010058200999999999,"objects_trimmed":0,"snaptrim_duration":0.00078961199999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":3950,"num_read_kb":15857,"num_write":467,"num_write_kb":7750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":4249683,"num_keys_recovered":143,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"167'432","reported_seq":1851,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:29.229192+0000","last_change":"2024-09-13T09:20:29.229428+0000","last_active":"2024-09-13T09:20:29.229192+0000","last_peered":"2024-09-13T09:20:29.229192+0000","last_clean":"2024-09-13T09:20:29.229192+0000","last_became_active":"2024-09-13T09:17:40.801195+0000","last_became_peered":"2024-09-13T09:17:40.801195+0000","last_unstale":"2024-09-13T09:20:29.229192+0000","last_undegraded":"2024-09-13T09:20:29.229192+0000","last_fullsized":"2024-09-13T09:20:29.229192+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'432","last_scrub_stamp":"2024-09-13T09:20:29.229135+0000","last_deep_scrub":"167'432","last_deep_scrub_stamp":"2024-09-13T09:20:29.229135+0000","last_clean_scrub_stamp":"2024-09-13T09:20:29.229135+0000","objects_scrubbed":3,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:51.153252+0000","scrub_duration":0.0090859649999999997,"objects_trimmed":0,"snaptrim_duration":0.001195911,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2626,"num_read_kb":8389,"num_write":522,"num_write_kb":11747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"149'683","reported_seq":1688,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.727484+0000","last_change":"2024-09-13T09:19:51.722579+0000","last_active":"2024-09-13T09:20:12.727484+0000","last_peered":"2024-09-13T09:20:12.727484+0000","last_clean":"2024-09-13T09:20:12.727484+0000","last_became_active":"2024-09-13T09:19:49.736505+0000","last_became_peered":"2024-09-13T09:19:49.736505+0000","last_unstale":"2024-09-13T09:20:12.727484+0000","last_undegraded":"2024-09-13T09:20:12.727484+0000","last_fullsized":"2024-09-13T09:20:12.727484+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00218943,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2689,"num_read_kb":8298,"num_write":946,"num_write_kb":9158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":43,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"167'695","reported_seq":3246,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:30.774226+0000","last_change":"2024-09-13T09:20:30.774467+0000","last_active":"2024-09-13T09:20:30.774226+0000","last_peered":"2024-09-13T09:20:30.774226+0000","last_clean":"2024-09-13T09:20:30.774226+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:20:30.774226+0000","last_undegraded":"2024-09-13T09:20:30.774226+0000","last_fullsized":"2024-09-13T09:20:30.774226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'695","last_scrub_stamp":"2024-09-13T09:20:30.774168+0000","last_deep_scrub":"167'695","last_deep_scrub_stamp":"2024-09-13T09:20:30.774168+0000","last_clean_scrub_stamp":"2024-09-13T09:20:30.774168+0000","objects_scrubbed":2,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:07.567176+0000","scrub_duration":0.0084296590000000008,"objects_trimmed":0,"snaptrim_duration":0.0035729350000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4417,"num_read_kb":31654,"num_write":734,"num_write_kb":30325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"146'383","reported_seq":1378,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.727672+0000","last_change":"2024-09-13T09:19:51.721061+0000","last_active":"2024-09-13T09:20:12.727672+0000","last_peered":"2024-09-13T09:20:12.727672+0000","last_clean":"2024-09-13T09:20:12.727672+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:20:12.727672+0000","last_undegraded":"2024-09-13T09:20:12.727672+0000","last_fullsized":"2024-09-13T09:20:12.727672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068946100000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1301,"num_read_kb":10607,"num_write":406,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"167'984","reported_seq":4069,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:33.731999+0000","last_change":"2024-09-13T09:20:33.732226+0000","last_active":"2024-09-13T09:20:33.731999+0000","last_peered":"2024-09-13T09:20:33.731999+0000","last_clean":"2024-09-13T09:20:33.731999+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:20:33.731999+0000","last_undegraded":"2024-09-13T09:20:33.731999+0000","last_fullsized":"2024-09-13T09:20:33.731999+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'984","last_scrub_stamp":"2024-09-13T09:20:33.731944+0000","last_deep_scrub":"167'984","last_deep_scrub_stamp":"2024-09-13T09:20:33.731944+0000","last_clean_scrub_stamp":"2024-09-13T09:20:33.731944+0000","objects_scrubbed":4,"log_size":984,"log_dups_size":0,"ondisk_log_size":984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:49.528565+0000","scrub_duration":0.011015433,"objects_trimmed":0,"snaptrim_duration":0.0024735249999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7129,"num_read_kb":22784,"num_write":1446,"num_write_kb":10311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"167'406","reported_seq":2324,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:22.301547+0000","last_change":"2024-09-13T09:20:22.301848+0000","last_active":"2024-09-13T09:20:22.301547+0000","last_peered":"2024-09-13T09:20:22.301547+0000","last_clean":"2024-09-13T09:20:22.301547+0000","last_became_active":"2024-09-13T09:19:20.803495+0000","last_became_peered":"2024-09-13T09:19:20.803495+0000","last_unstale":"2024-09-13T09:20:22.301547+0000","last_undegraded":"2024-09-13T09:20:22.301547+0000","last_fullsized":"2024-09-13T09:20:22.301547+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'406","last_scrub_stamp":"2024-09-13T09:20:22.301493+0000","last_deep_scrub":"167'406","last_deep_scrub_stamp":"2024-09-13T09:20:22.301493+0000","last_clean_scrub_stamp":"2024-09-13T09:20:22.301493+0000","objects_scrubbed":4,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:27.599633+0000","scrub_duration":0.010950427,"objects_trimmed":0,"snaptrim_duration":0.00072899800000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2117,"num_read_kb":10151,"num_write":496,"num_write_kb":12845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5102507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"167'553","reported_seq":3026,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:29.788358+0000","last_change":"2024-09-13T09:20:29.788575+0000","last_active":"2024-09-13T09:20:29.788358+0000","last_peered":"2024-09-13T09:20:29.788358+0000","last_clean":"2024-09-13T09:20:29.788358+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:20:29.788358+0000","last_undegraded":"2024-09-13T09:20:29.788358+0000","last_fullsized":"2024-09-13T09:20:29.788358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'553","last_scrub_stamp":"2024-09-13T09:20:29.788300+0000","last_deep_scrub":"167'553","last_deep_scrub_stamp":"2024-09-13T09:20:29.788300+0000","last_clean_scrub_stamp":"2024-09-13T09:20:29.788300+0000","objects_scrubbed":4,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:15.140691+0000","scrub_duration":0.010107418,"objects_trimmed":0,"snaptrim_duration":0.0020210359999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":14560,"num_write":713,"num_write_kb":15391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"167'442","reported_seq":4951,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:22.892931+0000","last_change":"2024-09-13T09:20:22.893154+0000","last_active":"2024-09-13T09:20:22.892931+0000","last_peered":"2024-09-13T09:20:22.892931+0000","last_clean":"2024-09-13T09:20:22.892931+0000","last_became_active":"2024-09-13T09:17:40.789499+0000","last_became_peered":"2024-09-13T09:17:40.789499+0000","last_unstale":"2024-09-13T09:20:22.892931+0000","last_undegraded":"2024-09-13T09:20:22.892931+0000","last_fullsized":"2024-09-13T09:20:22.892931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'442","last_scrub_stamp":"2024-09-13T09:20:22.892876+0000","last_deep_scrub":"167'442","last_deep_scrub_stamp":"2024-09-13T09:20:22.892876+0000","last_clean_scrub_stamp":"2024-09-13T09:20:22.892876+0000","objects_scrubbed":4,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:18.401596+0000","scrub_duration":0.0099034080000000007,"objects_trimmed":0,"snaptrim_duration":0.0010644070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5803,"num_read_kb":44996,"num_write":398,"num_write_kb":17054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"147'421","reported_seq":3621,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:12.727106+0000","last_change":"2024-09-13T09:19:51.723691+0000","last_active":"2024-09-13T09:20:12.727106+0000","last_peered":"2024-09-13T09:20:12.727106+0000","last_clean":"2024-09-13T09:20:12.727106+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:20:12.727106+0000","last_undegraded":"2024-09-13T09:20:12.727106+0000","last_fullsized":"2024-09-13T09:20:12.727106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:17:26.547451+0000","last_clean_scrub_stamp":"2024-09-13T09:17:26.547451+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003077476,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4122,"num_read_kb":18425,"num_write":408,"num_write_kb":9869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":152,"num_read":126915,"num_read_kb":555724,"num_write":19133,"num_write_kb":379149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1469,"num_bytes_recovered":28693609,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16819,"ondisk_log_size":16819,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":30095,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59372,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312468,"statfs":{"total":96636764160,"available":96575967232,"internally_reserved":0,"allocated":2068480,"data_stored":1548192,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.95699999999999996,"15min":0.89200000000000002},"min":{"1min":0.63500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.1799999999999999,"5min":11.683,"15min":11.683},"last":0.873},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.0269999999999999,"15min":0.98099999999999998},"min":{"1min":0.60699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2729999999999999,"5min":11.843999999999999,"15min":11.843999999999999},"last":1.026}]},{"osd":1,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.85699999999999998,"15min":0.86299999999999999},"min":{"1min":0.54900000000000004,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.175,"5min":1.286,"15min":1.286},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.995,"5min":0.95099999999999996,"15min":0.98299999999999998},"min":{"1min":0.61699999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.304,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":0.80800000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.82099999999999995,"15min":0.81000000000000005},"min":{"1min":0.53800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.73199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.109}]},{"osd":3,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.0289999999999999,"15min":0.97099999999999997},"min":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.597,"5min":12.012,"15min":12.012},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.139,"15min":1.077},"min":{"1min":0.66400000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.615,"5min":11.776999999999999,"15min":11.776999999999999},"last":0.997}]},{"osd":4,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.0049999999999999,"15min":1.0529999999999999},"min":{"1min":0.55700000000000005,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4319999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95799999999999996,"15min":0.96899999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.0529999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.079,"15min":1.073},"min":{"1min":0.59799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6990000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.175},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1160000000000001,"15min":1.143},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.651,"5min":3.125,"15min":3.125},"last":1.2450000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.163,"15min":1.222},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.9910000000000001,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.163,"5min":1.117,"15min":1.1220000000000001},"min":{"1min":0.52000000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4750000000000001,"5min":1.996,"15min":1.996},"last":1.1419999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41268,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330572,"statfs":{"total":96636764160,"available":96594505728,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.438,"5min":0.314,"15min":0.314},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.877},"min":{"1min":0.40899999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3580000000000001,"5min":2.222,"15min":2.222},"last":0.96299999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.83399999999999996,"15min":0.81200000000000006},"min":{"1min":0.50800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":1.121},{"interface":"front","average":{"1min":1.04,"5min":0.98199999999999998,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.607,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.2509999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.65700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4319999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.98499999999999999,"15min":1.0029999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.613,"5min":2.508,"15min":2.508},"last":0.83199999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.66900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.55,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.024},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.2450000000000001,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.996}]},{"osd":4,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.95199999999999996,"15min":0.97399999999999998},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2669999999999999,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.008,"15min":1},"min":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5800000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.052}]},{"osd":5,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.92300000000000004,"15min":0.90000000000000002},"min":{"1min":0.58999999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.57599999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.8120000000000001,"5min":2.286,"15min":2.286},"last":1.149}]},{"osd":7,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.093,"15min":1.087},"min":{"1min":0.621,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.611,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.079,"15min":1.0489999999999999},"min":{"1min":0.69899999999999995,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.208}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55224,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316616,"statfs":{"total":96636764160,"available":96580214784,"internally_reserved":0,"allocated":2015232,"data_stored":957843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":4.992,"15min":6.8739999999999997},"min":{"1min":0.47299999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.306,"5min":132.32900000000001,"15min":132.32900000000001},"last":1.077},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81499999999999995,"15min":0.81699999999999995},"min":{"1min":0.50900000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.151,"5min":1.335,"15min":1.335},"last":0.89400000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.84099999999999997,"15min":0.877},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2010000000000001,"5min":1.946,"15min":1.946},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97899999999999998,"15min":1.0149999999999999},"min":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4099999999999999,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.129}]},{"osd":2,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84199999999999997,"15min":0.84399999999999997},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.101},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.875,"15min":0.89300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2130000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.92900000000000005}]},{"osd":3,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.94399999999999995},"min":{"1min":0.61299999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.286,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.98399999999999999,"15min":1},"min":{"1min":0.60899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.476,"15min":1.476},"last":1.05}]},{"osd":4,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.88900000000000001,"15min":0.872},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.458,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.103,"5min":5.375,"15min":7.2999999999999998},"min":{"1min":0.55100000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.395,"5min":132.23599999999999,"15min":132.23599999999999},"last":0.96099999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.97099999999999997,"15min":0.96699999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.747},{"interface":"front","average":{"1min":0.997,"5min":1.0069999999999999,"15min":1.03},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.5820000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.016}]},{"osd":7,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.085,"15min":1.0780000000000001},"min":{"1min":0.67900000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.359,"5min":1.655,"15min":1.655},"last":1.175},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.115,"15min":1.131},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.4850000000000001,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51128,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320712,"statfs":{"total":96636764160,"available":96584409088,"internally_reserved":0,"allocated":2015232,"data_stored":957843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3560000000000001,"5min":1.357,"15min":1.357},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.94599999999999995,"15min":0.94799999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.1459999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91400000000000003,"15min":0.92600000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.01},{"interface":"front","average":{"1min":0.873,"5min":0.81100000000000005,"15min":0.81999999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.0429999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.877},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.006,"5min":0.88,"15min":0.84299999999999997},"min":{"1min":0.56999999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.76300000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86899999999999999,"15min":0.85399999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3300000000000001,"5min":1.575,"15min":1.575},"last":1.173},{"interface":"front","average":{"1min":1.069,"5min":1.0249999999999999,"15min":0.97799999999999998},"min":{"1min":0.628,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0899999999999999,"5min":4.7389999999999999,"15min":4.7389999999999999},"last":0.88200000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.45400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.083},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.94299999999999995}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0329999999999999},"min":{"1min":0.53500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.181,"5min":1.077,"15min":1.0629999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.4119999999999999,"5min":2.4119999999999999,"15min":2.4119999999999999},"last":0.89900000000000002}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.034,"15min":1.026},"min":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5609999999999999,"5min":1.597,"15min":1.597},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.125,"15min":1.131},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.024}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77364,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94294476,"statfs":{"total":96636764160,"available":96557543424,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.249,"5min":0.215,"15min":0.215},"max":{"1min":4.492,"5min":4.492,"15min":4.492},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.33100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.337,"5min":1.625,"15min":1.625},"last":0.90500000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.38600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.4990000000000001,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.72099999999999997,"15min":0.68899999999999995},"min":{"1min":0.29299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.3890000000000002,"5min":4.3890000000000002,"15min":4.3890000000000002},"last":0.67700000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.73099999999999998,"15min":0.70299999999999996},"min":{"1min":0.44,"5min":0.312,"15min":0.312},"max":{"1min":1.45,"5min":1.544,"15min":1.544},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.79600000000000004,"15min":0.77600000000000002},"min":{"1min":0.41699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.2880000000000003,"5min":4.2880000000000003,"15min":4.2880000000000003},"last":1.153}]},{"osd":4,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.90400000000000003,"15min":0.91000000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.25},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.94899999999999995,"15min":0.94699999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.86099999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0049999999999999,"15min":0.98299999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":5.3319999999999999,"5min":5.3319999999999999,"15min":5.3319999999999999},"last":0.997},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.93899999999999995,"15min":0.93700000000000006},"min":{"1min":0.55200000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.1950000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.022,"15min":1.0289999999999999},"min":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.272,"5min":1.1200000000000001,"15min":1.1080000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":5.25,"5min":5.25,"15min":5.25},"last":0.83599999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.0149999999999999,"15min":1.012},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.6899999999999999,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":1.109},{"interface":"front","average":{"1min":1.137,"5min":1.5269999999999999,"15min":1.726},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.504},"max":{"1min":2.097,"5min":15.4,"15min":15.4},"last":1.0860000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41268,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330572,"statfs":{"total":96636764160,"available":96594505728,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.621},"min":{"1min":0.247,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1899999999999999,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.72499999999999998,"15min":0.70499999999999996},"min":{"1min":0.28399999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.671,"5min":2.8490000000000002,"15min":2.8490000000000002},"last":0.62}]},{"osd":1,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.79000000000000004,"15min":0.79300000000000004},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.337,"5min":1.907,"15min":1.907},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.77600000000000002,"15min":0.74399999999999999},"min":{"1min":0.48999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.256,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.83899999999999997,"15min":0.871},"min":{"1min":0.34000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.47,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.97399999999999998,"15min":0.96999999999999997},"min":{"1min":0.76200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4470000000000001,"5min":1.782,"15min":1.782},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.83499999999999996,"15min":0.81699999999999995},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.1439999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.97999999999999998,"15min":0.96799999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0329999999999999,"15min":1.0289999999999999},"min":{"1min":0.57199999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5580000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.1870000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.55600000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.012,"15min":1.0049999999999999},"min":{"1min":0.623,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.923,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.98299999999999998}]},{"osd":7,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.121,"15min":1.133},"min":{"1min":0.68899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.573,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.0049999999999999,"15min":1.0029999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5620000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.0249999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":2019328,"data_stored":957862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.623},"min":{"1min":0.35799999999999998,"5min":0.255,"15min":0.255},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.34300000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.95499999999999996,"5min":1.484,"15min":1.484},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72899999999999998,"15min":0.75700000000000001},"min":{"1min":0.36699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.0569999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.83199999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.753},"min":{"1min":0.44600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.135,"5min":1.264,"15min":1.264},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76500000000000001,"15min":0.76900000000000002},"min":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1499999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.79500000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3080000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.749},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97699999999999998,"15min":1.0109999999999999},"min":{"1min":0.64800000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3620000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.94599999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0780000000000001,"15min":1.0409999999999999},"min":{"1min":0.60799999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.524,"5min":8.3460000000000001,"15min":8.3460000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98799999999999999,"15min":1.018},"min":{"1min":0.56599999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4710000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.0669999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.01,"15min":1.044},"min":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.573,"5min":1.647,"15min":1.647},"last":1.05},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.002,"15min":1.028},"min":{"1min":0.61499999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.996}]},{"osd":7,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.024,"15min":1.05},"min":{"1min":0.69099999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5800000000000001,"5min":1.629,"15min":1.629},"last":1.024},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.081,"15min":1.1140000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.97799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33116,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":2052096,"data_stored":1548192,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.316,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.325,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.625,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.006,"5min":1.284,"15min":1.284},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92600000000000005,"5min":1.5,"15min":1.5},"last":0.745},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.39000000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.129,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.623}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.72499999999999998,"15min":0.69899999999999995},"min":{"1min":0.42699999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.452,"5min":2.242,"15min":2.242},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.78900000000000003},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2250000000000001,"5min":1.528,"15min":1.528},"last":0.89100000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.89200000000000002,"15min":0.877},"min":{"1min":0.53700000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0309999999999999,"15min":0.96599999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.484,"5min":9.5649999999999995,"15min":9.5649999999999995},"last":1.131}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0469999999999999,"15min":0.95799999999999996},"min":{"1min":0.67900000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.375,"5min":9.6120000000000001,"15min":9.6120000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.419,"5min":1.476,"15min":1.476},"last":0.95999999999999996}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0620000000000001,"15min":1.085},"min":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4279999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.996,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.482,"5min":1.516,"15min":1.516},"last":1.204}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0229999999999999,"15min":1.0309999999999999},"min":{"1min":0.58799999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.5669999999999999,"5min":1.617,"15min":1.617},"last":1.2849999999999999},{"interface":"front","average":{"1min":0.995,"5min":1.1379999999999999,"15min":1.0820000000000001},"min":{"1min":0.67300000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.546,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":1.2490000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:20:42.751 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:42.751 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:42.751 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:42.751 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T09:17:26.547451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=17, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=20, tm_sec=17, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:20:42.751 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T09:21:02.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:21:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-13T09:21:03.067 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-13T09:21:03.084 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-13T09:21:02.120307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":152,"num_read":127031,"num_read_kb":555824,"num_write":19294,"num_write_kb":381567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1476,"num_bytes_recovered":29874402,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16964,"ondisk_log_size":16964,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":391824,"kb_used_data":15824,"kb_used_omap":12,"kb_used_meta":375923,"kb_avail":754582896,"statfs":{"total":773094113280,"available":772692885504,"internally_reserved":0,"allocated":16203776,"data_stored":8843404,"data_compressed":19717,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12744,"internal_metadata":384945720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003797"},"pg_stats":[{"pgid":"3.1f","version":"167'408","reported_seq":1897,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:30.988502+0000","last_change":"2024-09-13T09:20:30.988734+0000","last_active":"2024-09-13T09:20:30.988502+0000","last_peered":"2024-09-13T09:20:30.988502+0000","last_clean":"2024-09-13T09:20:30.988502+0000","last_became_active":"2024-09-13T09:17:40.791187+0000","last_became_peered":"2024-09-13T09:17:40.791187+0000","last_unstale":"2024-09-13T09:20:30.988502+0000","last_undegraded":"2024-09-13T09:20:30.988502+0000","last_fullsized":"2024-09-13T09:20:30.988502+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'408","last_scrub_stamp":"2024-09-13T09:20:30.988449+0000","last_deep_scrub":"167'408","last_deep_scrub_stamp":"2024-09-13T09:20:30.988449+0000","last_clean_scrub_stamp":"2024-09-13T09:20:30.988449+0000","objects_scrubbed":2,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:28:22.162575+0000","scrub_duration":0.0078384240000000001,"objects_trimmed":0,"snaptrim_duration":0.001951139,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2539,"num_read_kb":13964,"num_write":479,"num_write_kb":12323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"167'646","reported_seq":3301,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:22.201025+0000","last_change":"2024-09-13T09:20:22.201131+0000","last_active":"2024-09-13T09:20:22.201025+0000","last_peered":"2024-09-13T09:20:22.201025+0000","last_clean":"2024-09-13T09:20:22.201025+0000","last_became_active":"2024-09-13T09:19:20.802379+0000","last_became_peered":"2024-09-13T09:19:20.802379+0000","last_unstale":"2024-09-13T09:20:22.201025+0000","last_undegraded":"2024-09-13T09:20:22.201025+0000","last_fullsized":"2024-09-13T09:20:22.201025+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'646","last_scrub_stamp":"2024-09-13T09:20:22.201003+0000","last_deep_scrub":"167'646","last_deep_scrub_stamp":"2024-09-13T09:20:22.201003+0000","last_clean_scrub_stamp":"2024-09-13T09:20:22.201003+0000","objects_scrubbed":6,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:37:35.776557+0000","scrub_duration":0.010201534,"objects_trimmed":0,"snaptrim_duration":0.001400552,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3630,"num_read_kb":10038,"num_write":856,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":8249616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"167'493","reported_seq":3779,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:37.041436+0000","last_change":"2024-09-13T09:20:37.041704+0000","last_active":"2024-09-13T09:20:37.041436+0000","last_peered":"2024-09-13T09:20:37.041436+0000","last_clean":"2024-09-13T09:20:37.041436+0000","last_became_active":"2024-09-13T09:17:26.708645+0000","last_became_peered":"2024-09-13T09:17:26.708645+0000","last_unstale":"2024-09-13T09:20:37.041436+0000","last_undegraded":"2024-09-13T09:20:37.041436+0000","last_fullsized":"2024-09-13T09:20:37.041436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'493","last_scrub_stamp":"2024-09-13T09:20:37.041380+0000","last_deep_scrub":"167'493","last_deep_scrub_stamp":"2024-09-13T09:20:37.041380+0000","last_clean_scrub_stamp":"2024-09-13T09:20:37.041380+0000","objects_scrubbed":5,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:34.537298+0000","scrub_duration":0.012442204,"objects_trimmed":0,"snaptrim_duration":0.0021011329999999998,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5768,"num_read_kb":20052,"num_write":439,"num_write_kb":19543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"167'796","reported_seq":4580,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:36.013488+0000","last_change":"2024-09-13T09:20:36.013737+0000","last_active":"2024-09-13T09:20:36.013488+0000","last_peered":"2024-09-13T09:20:36.013488+0000","last_clean":"2024-09-13T09:20:36.013488+0000","last_became_active":"2024-09-13T09:17:26.701439+0000","last_became_peered":"2024-09-13T09:17:26.701439+0000","last_unstale":"2024-09-13T09:20:36.013488+0000","last_undegraded":"2024-09-13T09:20:36.013488+0000","last_fullsized":"2024-09-13T09:20:36.013488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'796","last_scrub_stamp":"2024-09-13T09:20:36.013433+0000","last_deep_scrub":"167'796","last_deep_scrub_stamp":"2024-09-13T09:20:36.013433+0000","last_clean_scrub_stamp":"2024-09-13T09:20:36.013433+0000","objects_scrubbed":3,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:44.935327+0000","scrub_duration":0.010464815000000001,"objects_trimmed":0,"snaptrim_duration":0.0025684319999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":7779,"num_read_kb":18614,"num_write":881,"num_write_kb":10036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"167'533","reported_seq":4445,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:26.898210+0000","last_change":"2024-09-13T09:20:26.898292+0000","last_active":"2024-09-13T09:20:26.898210+0000","last_peered":"2024-09-13T09:20:26.898210+0000","last_clean":"2024-09-13T09:20:26.898210+0000","last_became_active":"2024-09-13T09:17:40.799618+0000","last_became_peered":"2024-09-13T09:17:40.799618+0000","last_unstale":"2024-09-13T09:20:26.898210+0000","last_undegraded":"2024-09-13T09:20:26.898210+0000","last_fullsized":"2024-09-13T09:20:26.898210+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'533","last_scrub_stamp":"2024-09-13T09:20:26.898189+0000","last_deep_scrub":"167'533","last_deep_scrub_stamp":"2024-09-13T09:20:26.898189+0000","last_clean_scrub_stamp":"2024-09-13T09:20:26.898189+0000","objects_scrubbed":2,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:18:27.402049+0000","scrub_duration":0.0061538280000000001,"objects_trimmed":0,"snaptrim_duration":0.001823089,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5766,"num_read_kb":19616,"num_write":659,"num_write_kb":9141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":65341,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"167'352","reported_seq":3308,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:26.894646+0000","last_change":"2024-09-13T09:20:26.894914+0000","last_active":"2024-09-13T09:20:26.894646+0000","last_peered":"2024-09-13T09:20:26.894646+0000","last_clean":"2024-09-13T09:20:26.894646+0000","last_became_active":"2024-09-13T09:18:03.973063+0000","last_became_peered":"2024-09-13T09:18:03.973063+0000","last_unstale":"2024-09-13T09:20:26.894646+0000","last_undegraded":"2024-09-13T09:20:26.894646+0000","last_fullsized":"2024-09-13T09:20:26.894646+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'352","last_scrub_stamp":"2024-09-13T09:20:26.894594+0000","last_deep_scrub":"167'352","last_deep_scrub_stamp":"2024-09-13T09:20:26.894594+0000","last_clean_scrub_stamp":"2024-09-13T09:20:26.894594+0000","objects_scrubbed":8,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:58.666673+0000","scrub_duration":0.013708579,"objects_trimmed":0,"snaptrim_duration":0.00082503800000000005,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3197,"num_read_kb":8961,"num_write":344,"num_write_kb":6710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":889520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"167'725","reported_seq":4273,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:35.813651+0000","last_change":"2024-09-13T09:20:35.813923+0000","last_active":"2024-09-13T09:20:35.813651+0000","last_peered":"2024-09-13T09:20:35.813651+0000","last_clean":"2024-09-13T09:20:35.813651+0000","last_became_active":"2024-09-13T09:18:06.975016+0000","last_became_peered":"2024-09-13T09:18:06.975016+0000","last_unstale":"2024-09-13T09:20:35.813651+0000","last_undegraded":"2024-09-13T09:20:35.813651+0000","last_fullsized":"2024-09-13T09:20:35.813651+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'725","last_scrub_stamp":"2024-09-13T09:20:35.813596+0000","last_deep_scrub":"167'725","last_deep_scrub_stamp":"2024-09-13T09:20:35.813596+0000","last_clean_scrub_stamp":"2024-09-13T09:20:35.813596+0000","objects_scrubbed":10,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:48.519519+0000","scrub_duration":0.021453799999999999,"objects_trimmed":0,"snaptrim_duration":0.0010502549999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5493,"num_read_kb":28565,"num_write":1011,"num_write_kb":11903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":439303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"167'223","reported_seq":2579,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:36.085607+0000","last_change":"2024-09-13T09:20:36.085832+0000","last_active":"2024-09-13T09:20:36.085607+0000","last_peered":"2024-09-13T09:20:36.085607+0000","last_clean":"2024-09-13T09:20:36.085607+0000","last_became_active":"2024-09-13T09:17:40.797607+0000","last_became_peered":"2024-09-13T09:17:40.797607+0000","last_unstale":"2024-09-13T09:20:36.085607+0000","last_undegraded":"2024-09-13T09:20:36.085607+0000","last_fullsized":"2024-09-13T09:20:36.085607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'223","last_scrub_stamp":"2024-09-13T09:20:36.085553+0000","last_deep_scrub":"167'223","last_deep_scrub_stamp":"2024-09-13T09:20:36.085553+0000","last_clean_scrub_stamp":"2024-09-13T09:20:36.085553+0000","objects_scrubbed":6,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:44:53.450646+0000","scrub_duration":0.012966777000000001,"objects_trimmed":0,"snaptrim_duration":0.0011389530000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2065,"num_read_kb":17454,"num_write":181,"num_write_kb":10975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":42856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"167'445","reported_seq":2316,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:19.859558+0000","last_change":"2024-09-13T09:20:19.859660+0000","last_active":"2024-09-13T09:20:19.859558+0000","last_peered":"2024-09-13T09:20:19.859558+0000","last_clean":"2024-09-13T09:20:19.859558+0000","last_became_active":"2024-09-13T09:17:43.809533+0000","last_became_peered":"2024-09-13T09:17:43.809533+0000","last_unstale":"2024-09-13T09:20:19.859558+0000","last_undegraded":"2024-09-13T09:20:19.859558+0000","last_fullsized":"2024-09-13T09:20:19.859558+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'445","last_scrub_stamp":"2024-09-13T09:20:19.859539+0000","last_deep_scrub":"167'445","last_deep_scrub_stamp":"2024-09-13T09:20:19.859539+0000","last_clean_scrub_stamp":"2024-09-13T09:20:19.859539+0000","objects_scrubbed":3,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:08:48.625415+0000","scrub_duration":0.0070617070000000004,"objects_trimmed":0,"snaptrim_duration":0.0018671040000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3080,"num_read_kb":24506,"num_write":379,"num_write_kb":12955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":16384,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"167'278","reported_seq":2302,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:35.003177+0000","last_change":"2024-09-13T09:20:35.003481+0000","last_active":"2024-09-13T09:20:35.003177+0000","last_peered":"2024-09-13T09:20:35.003177+0000","last_clean":"2024-09-13T09:20:35.003177+0000","last_became_active":"2024-09-13T09:17:26.713425+0000","last_became_peered":"2024-09-13T09:17:26.713425+0000","last_unstale":"2024-09-13T09:20:35.003177+0000","last_undegraded":"2024-09-13T09:20:35.003177+0000","last_fullsized":"2024-09-13T09:20:35.003177+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'278","last_scrub_stamp":"2024-09-13T09:20:35.003118+0000","last_deep_scrub":"167'278","last_deep_scrub_stamp":"2024-09-13T09:20:35.003118+0000","last_clean_scrub_stamp":"2024-09-13T09:20:35.003118+0000","objects_scrubbed":6,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:55:51.435272+0000","scrub_duration":0.013897542000000001,"objects_trimmed":0,"snaptrim_duration":0.001510742,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1901,"num_read_kb":15086,"num_write":247,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"167'374","reported_seq":2983,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:54.848001+0000","last_change":"2024-09-13T09:20:54.848225+0000","last_active":"2024-09-13T09:20:54.848001+0000","last_peered":"2024-09-13T09:20:54.848001+0000","last_clean":"2024-09-13T09:20:54.848001+0000","last_became_active":"2024-09-13T09:17:26.704595+0000","last_became_peered":"2024-09-13T09:17:26.704595+0000","last_unstale":"2024-09-13T09:20:54.848001+0000","last_undegraded":"2024-09-13T09:20:54.848001+0000","last_fullsized":"2024-09-13T09:20:54.848001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'374","last_scrub_stamp":"2024-09-13T09:20:54.847942+0000","last_deep_scrub":"167'374","last_deep_scrub_stamp":"2024-09-13T09:20:54.847942+0000","last_clean_scrub_stamp":"2024-09-13T09:20:54.847942+0000","objects_scrubbed":3,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:19:08.661607+0000","scrub_duration":0.0095540839999999991,"objects_trimmed":0,"snaptrim_duration":0.0029658509999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3481,"num_read_kb":18288,"num_write":412,"num_write_kb":15933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"167'584","reported_seq":3095,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:29.876176+0000","last_change":"2024-09-13T09:20:29.876393+0000","last_active":"2024-09-13T09:20:29.876176+0000","last_peered":"2024-09-13T09:20:29.876176+0000","last_clean":"2024-09-13T09:20:29.876176+0000","last_became_active":"2024-09-13T09:17:26.700923+0000","last_became_peered":"2024-09-13T09:17:26.700923+0000","last_unstale":"2024-09-13T09:20:29.876176+0000","last_undegraded":"2024-09-13T09:20:29.876176+0000","last_fullsized":"2024-09-13T09:20:29.876176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'584","last_scrub_stamp":"2024-09-13T09:20:29.876121+0000","last_deep_scrub":"167'584","last_deep_scrub_stamp":"2024-09-13T09:20:29.876121+0000","last_clean_scrub_stamp":"2024-09-13T09:20:29.876121+0000","objects_scrubbed":4,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:49.387878+0000","scrub_duration":0.011138855,"objects_trimmed":0,"snaptrim_duration":0.001048134,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3471,"num_read_kb":17629,"num_write":764,"num_write_kb":18883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:24.305923+0000","last_change":"2024-09-13T09:20:24.306194+0000","last_active":"2024-09-13T09:20:24.305923+0000","last_peered":"2024-09-13T09:20:24.305923+0000","last_clean":"2024-09-13T09:20:24.305923+0000","last_became_active":"2024-09-13T09:19:17.796177+0000","last_became_peered":"2024-09-13T09:19:17.796177+0000","last_unstale":"2024-09-13T09:20:24.305923+0000","last_undegraded":"2024-09-13T09:20:24.305923+0000","last_fullsized":"2024-09-13T09:20:24.305923+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:20:24.305847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:20:24.305847+0000","last_clean_scrub_stamp":"2024-09-13T09:20:24.305847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:51:09.170716+0000","scrub_duration":0.0030267710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"167'454","reported_seq":1845,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:32.020794+0000","last_change":"2024-09-13T09:20:32.021125+0000","last_active":"2024-09-13T09:20:32.020794+0000","last_peered":"2024-09-13T09:20:32.020794+0000","last_clean":"2024-09-13T09:20:32.020794+0000","last_became_active":"2024-09-13T09:19:14.748293+0000","last_became_peered":"2024-09-13T09:19:14.748293+0000","last_unstale":"2024-09-13T09:20:32.020794+0000","last_undegraded":"2024-09-13T09:20:32.020794+0000","last_fullsized":"2024-09-13T09:20:32.020794+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'454","last_scrub_stamp":"2024-09-13T09:20:32.020683+0000","last_deep_scrub":"167'454","last_deep_scrub_stamp":"2024-09-13T09:20:32.020683+0000","last_clean_scrub_stamp":"2024-09-13T09:20:32.020683+0000","objects_scrubbed":4,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:58:59.331014+0000","scrub_duration":0.010434268999999999,"objects_trimmed":0,"snaptrim_duration":0.0016656990000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2578,"num_read_kb":17393,"num_write":357,"num_write_kb":10507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":7471802,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":157,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:28.771225+0000","last_change":"2024-09-13T09:20:28.771488+0000","last_active":"2024-09-13T09:20:28.771225+0000","last_peered":"2024-09-13T09:20:28.771225+0000","last_clean":"2024-09-13T09:20:28.771225+0000","last_became_active":"2024-09-13T09:19:17.769570+0000","last_became_peered":"2024-09-13T09:19:17.769570+0000","last_unstale":"2024-09-13T09:20:28.771225+0000","last_undegraded":"2024-09-13T09:20:28.771225+0000","last_fullsized":"2024-09-13T09:20:28.771225+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:20:28.771166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:20:28.771166+0000","last_clean_scrub_stamp":"2024-09-13T09:20:28.771166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:43.434175+0000","scrub_duration":0.0032507159999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"167'565","reported_seq":3028,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:28.188435+0000","last_change":"2024-09-13T09:20:28.188667+0000","last_active":"2024-09-13T09:20:28.188435+0000","last_peered":"2024-09-13T09:20:28.188435+0000","last_clean":"2024-09-13T09:20:28.188435+0000","last_became_active":"2024-09-13T09:17:26.702257+0000","last_became_peered":"2024-09-13T09:17:26.702257+0000","last_unstale":"2024-09-13T09:20:28.188435+0000","last_undegraded":"2024-09-13T09:20:28.188435+0000","last_fullsized":"2024-09-13T09:20:28.188435+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'565","last_scrub_stamp":"2024-09-13T09:20:28.188378+0000","last_deep_scrub":"167'565","last_deep_scrub_stamp":"2024-09-13T09:20:28.188378+0000","last_clean_scrub_stamp":"2024-09-13T09:20:28.188378+0000","objects_scrubbed":4,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:13.700620+0000","scrub_duration":0.010427054999999999,"objects_trimmed":0,"snaptrim_duration":0.0015869739999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4402,"num_read_kb":8559,"num_write":727,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":189,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:27.775438+0000","last_change":"2024-09-13T09:20:27.775674+0000","last_active":"2024-09-13T09:20:27.775438+0000","last_peered":"2024-09-13T09:20:27.775438+0000","last_clean":"2024-09-13T09:20:27.775438+0000","last_became_active":"2024-09-13T09:17:40.793882+0000","last_became_peered":"2024-09-13T09:17:40.793882+0000","last_unstale":"2024-09-13T09:20:27.775438+0000","last_undegraded":"2024-09-13T09:20:27.775438+0000","last_fullsized":"2024-09-13T09:20:27.775438+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:20:27.775385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:20:27.775385+0000","last_clean_scrub_stamp":"2024-09-13T09:20:27.775385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:01.437595+0000","scrub_duration":0.003321231,"objects_trimmed":0,"snaptrim_duration":0.00091874899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"167'385","reported_seq":1642,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:31.759985+0000","last_change":"2024-09-13T09:20:31.760219+0000","last_active":"2024-09-13T09:20:31.759985+0000","last_peered":"2024-09-13T09:20:31.759985+0000","last_clean":"2024-09-13T09:20:31.759985+0000","last_became_active":"2024-09-13T09:18:03.973314+0000","last_became_peered":"2024-09-13T09:18:03.973314+0000","last_unstale":"2024-09-13T09:20:31.759985+0000","last_undegraded":"2024-09-13T09:20:31.759985+0000","last_fullsized":"2024-09-13T09:20:31.759985+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'385","last_scrub_stamp":"2024-09-13T09:20:31.759894+0000","last_deep_scrub":"167'385","last_deep_scrub_stamp":"2024-09-13T09:20:31.759894+0000","last_clean_scrub_stamp":"2024-09-13T09:20:31.759894+0000","objects_scrubbed":8,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:31:47.761295+0000","scrub_duration":0.015260414999999999,"objects_trimmed":0,"snaptrim_duration":0.0012376119999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1399,"num_read_kb":26203,"num_write":287,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":167476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:34.773034+0000","last_change":"2024-09-13T09:20:34.773326+0000","last_active":"2024-09-13T09:20:34.773034+0000","last_peered":"2024-09-13T09:20:34.773034+0000","last_clean":"2024-09-13T09:20:34.773034+0000","last_became_active":"2024-09-13T09:18:03.967133+0000","last_became_peered":"2024-09-13T09:18:03.967133+0000","last_unstale":"2024-09-13T09:20:34.773034+0000","last_undegraded":"2024-09-13T09:20:34.773034+0000","last_fullsized":"2024-09-13T09:20:34.773034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:20:34.772980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:20:34.772980+0000","last_clean_scrub_stamp":"2024-09-13T09:20:34.772980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:18:21.081636+0000","scrub_duration":0.0031266240000000002,"objects_trimmed":0,"snaptrim_duration":0.000915543,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"167'349","reported_seq":2285,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:34.010284+0000","last_change":"2024-09-13T09:20:34.010523+0000","last_active":"2024-09-13T09:20:34.010284+0000","last_peered":"2024-09-13T09:20:34.010284+0000","last_clean":"2024-09-13T09:20:34.010284+0000","last_became_active":"2024-09-13T09:17:26.708604+0000","last_became_peered":"2024-09-13T09:17:26.708604+0000","last_unstale":"2024-09-13T09:20:34.010284+0000","last_undegraded":"2024-09-13T09:20:34.010284+0000","last_fullsized":"2024-09-13T09:20:34.010284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'349","last_scrub_stamp":"2024-09-13T09:20:34.010225+0000","last_deep_scrub":"167'349","last_deep_scrub_stamp":"2024-09-13T09:20:34.010225+0000","last_clean_scrub_stamp":"2024-09-13T09:20:34.010225+0000","objects_scrubbed":5,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:59:37.255952+0000","scrub_duration":0.01062372,"objects_trimmed":0,"snaptrim_duration":0.00109892,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2142,"num_read_kb":9020,"num_write":356,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"167'659","reported_seq":4767,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:25.868457+0000","last_change":"2024-09-13T09:20:25.868691+0000","last_active":"2024-09-13T09:20:25.868457+0000","last_peered":"2024-09-13T09:20:25.868457+0000","last_clean":"2024-09-13T09:20:25.868457+0000","last_became_active":"2024-09-13T09:17:40.800027+0000","last_became_peered":"2024-09-13T09:17:40.800027+0000","last_unstale":"2024-09-13T09:20:25.868457+0000","last_undegraded":"2024-09-13T09:20:25.868457+0000","last_fullsized":"2024-09-13T09:20:25.868457+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'659","last_scrub_stamp":"2024-09-13T09:20:25.868397+0000","last_deep_scrub":"167'659","last_deep_scrub_stamp":"2024-09-13T09:20:25.868397+0000","last_clean_scrub_stamp":"2024-09-13T09:20:25.868397+0000","objects_scrubbed":4,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:14.674500+0000","scrub_duration":0.010044911,"objects_trimmed":0,"snaptrim_duration":0.00072429099999999995,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":5951,"num_read_kb":14391,"num_write":711,"num_write_kb":6974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":29248,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"167'3","reported_seq":212,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:25.937343+0000","last_change":"2024-09-13T09:20:25.937597+0000","last_active":"2024-09-13T09:20:25.937343+0000","last_peered":"2024-09-13T09:20:25.937343+0000","last_clean":"2024-09-13T09:20:25.937343+0000","last_became_active":"2024-09-13T09:18:03.970835+0000","last_became_peered":"2024-09-13T09:18:03.970835+0000","last_unstale":"2024-09-13T09:20:25.937343+0000","last_undegraded":"2024-09-13T09:20:25.937343+0000","last_fullsized":"2024-09-13T09:20:25.937343+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'3","last_scrub_stamp":"2024-09-13T09:20:25.937291+0000","last_deep_scrub":"167'3","last_deep_scrub_stamp":"2024-09-13T09:20:25.937291+0000","last_clean_scrub_stamp":"2024-09-13T09:20:25.937291+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:43.100284+0000","scrub_duration":0.0060844330000000002,"objects_trimmed":0,"snaptrim_duration":0.0024130810000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"167'565","reported_seq":2696,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:33.758728+0000","last_change":"2024-09-13T09:20:33.758978+0000","last_active":"2024-09-13T09:20:33.758728+0000","last_peered":"2024-09-13T09:20:33.758728+0000","last_clean":"2024-09-13T09:20:33.758728+0000","last_became_active":"2024-09-13T09:18:03.970479+0000","last_became_peered":"2024-09-13T09:18:03.970479+0000","last_unstale":"2024-09-13T09:20:33.758728+0000","last_undegraded":"2024-09-13T09:20:33.758728+0000","last_fullsized":"2024-09-13T09:20:33.758728+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'565","last_scrub_stamp":"2024-09-13T09:20:33.758671+0000","last_deep_scrub":"167'565","last_deep_scrub_stamp":"2024-09-13T09:20:33.758671+0000","last_clean_scrub_stamp":"2024-09-13T09:20:33.758671+0000","objects_scrubbed":4,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:42:11.958274+0000","scrub_duration":0.010858072,"objects_trimmed":0,"snaptrim_duration":0.001031719,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5538,"num_read_kb":12570,"num_write":635,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1031390,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:21.445039+0000","last_change":"2024-09-13T09:20:21.445137+0000","last_active":"2024-09-13T09:20:21.445039+0000","last_peered":"2024-09-13T09:20:21.445039+0000","last_clean":"2024-09-13T09:20:21.445039+0000","last_became_active":"2024-09-13T09:20:02.756499+0000","last_became_peered":"2024-09-13T09:20:02.756499+0000","last_unstale":"2024-09-13T09:20:21.445039+0000","last_undegraded":"2024-09-13T09:20:21.445039+0000","last_fullsized":"2024-09-13T09:20:21.445039+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:20:21.445020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:20:21.445020+0000","last_clean_scrub_stamp":"2024-09-13T09:20:21.445020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:38:10.626621+0000","scrub_duration":0.0013369810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"167'490","reported_seq":3465,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:20.907209+0000","last_change":"2024-09-13T09:20:20.907319+0000","last_active":"2024-09-13T09:20:20.907209+0000","last_peered":"2024-09-13T09:20:20.907209+0000","last_clean":"2024-09-13T09:20:20.907209+0000","last_became_active":"2024-09-13T09:17:40.793065+0000","last_became_peered":"2024-09-13T09:17:40.793065+0000","last_unstale":"2024-09-13T09:20:20.907209+0000","last_undegraded":"2024-09-13T09:20:20.907209+0000","last_fullsized":"2024-09-13T09:20:20.907209+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'490","last_scrub_stamp":"2024-09-13T09:20:20.907181+0000","last_deep_scrub":"167'490","last_deep_scrub_stamp":"2024-09-13T09:20:20.907181+0000","last_clean_scrub_stamp":"2024-09-13T09:20:20.907181+0000","objects_scrubbed":7,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:09.277243+0000","scrub_duration":0.01046126,"objects_trimmed":0,"snaptrim_duration":0.00175362,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4025,"num_read_kb":26901,"num_write":443,"num_write_kb":18254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":127926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":190,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:24.778333+0000","last_change":"2024-09-13T09:20:24.778605+0000","last_active":"2024-09-13T09:20:24.778333+0000","last_peered":"2024-09-13T09:20:24.778333+0000","last_clean":"2024-09-13T09:20:24.778333+0000","last_became_active":"2024-09-13T09:18:03.968092+0000","last_became_peered":"2024-09-13T09:18:03.968092+0000","last_unstale":"2024-09-13T09:20:24.778333+0000","last_undegraded":"2024-09-13T09:20:24.778333+0000","last_fullsized":"2024-09-13T09:20:24.778333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:20:24.778284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:20:24.778284+0000","last_clean_scrub_stamp":"2024-09-13T09:20:24.778284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:36:17.198676+0000","scrub_duration":0.0033279809999999998,"objects_trimmed":0,"snaptrim_duration":0.0008384,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"167'790","reported_seq":2807,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:27.243135+0000","last_change":"2024-09-13T09:20:27.243385+0000","last_active":"2024-09-13T09:20:27.243135+0000","last_peered":"2024-09-13T09:20:27.243135+0000","last_clean":"2024-09-13T09:20:27.243135+0000","last_became_active":"2024-09-13T09:17:26.699489+0000","last_became_peered":"2024-09-13T09:17:26.699489+0000","last_unstale":"2024-09-13T09:20:27.243135+0000","last_undegraded":"2024-09-13T09:20:27.243135+0000","last_fullsized":"2024-09-13T09:20:27.243135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'790","last_scrub_stamp":"2024-09-13T09:20:27.243075+0000","last_deep_scrub":"167'790","last_deep_scrub_stamp":"2024-09-13T09:20:27.243075+0000","last_clean_scrub_stamp":"2024-09-13T09:20:27.243075+0000","objects_scrubbed":8,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:54:21.889458+0000","scrub_duration":0.018141651000000002,"objects_trimmed":0,"snaptrim_duration":0.0012273519999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4598,"num_read_kb":13738,"num_write":1068,"num_write_kb":11812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"167'2","reported_seq":175,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:29.003759+0000","last_change":"2024-09-13T09:20:29.004066+0000","last_active":"2024-09-13T09:20:29.003759+0000","last_peered":"2024-09-13T09:20:29.003759+0000","last_clean":"2024-09-13T09:20:29.003759+0000","last_became_active":"2024-09-13T09:17:17.495738+0000","last_became_peered":"2024-09-13T09:17:17.495738+0000","last_unstale":"2024-09-13T09:20:29.003759+0000","last_undegraded":"2024-09-13T09:20:29.003759+0000","last_fullsized":"2024-09-13T09:20:29.003759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'2","last_scrub_stamp":"2024-09-13T09:20:29.003640+0000","last_deep_scrub":"167'2","last_deep_scrub_stamp":"2024-09-13T09:20:29.003640+0000","last_clean_scrub_stamp":"2024-09-13T09:20:29.003640+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:40:04.692771+0000","scrub_duration":0.0064562659999999996,"objects_trimmed":0,"snaptrim_duration":0.00089519600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"167'119","reported_seq":366,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:22.796156+0000","last_change":"2024-09-13T09:20:22.796429+0000","last_active":"2024-09-13T09:20:22.796156+0000","last_peered":"2024-09-13T09:20:22.796156+0000","last_clean":"2024-09-13T09:20:22.796156+0000","last_became_active":"2024-09-13T09:17:43.802603+0000","last_became_peered":"2024-09-13T09:17:43.802603+0000","last_unstale":"2024-09-13T09:20:22.796156+0000","last_undegraded":"2024-09-13T09:20:22.796156+0000","last_fullsized":"2024-09-13T09:20:22.796156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'119","last_scrub_stamp":"2024-09-13T09:20:22.796093+0000","last_deep_scrub":"167'119","last_deep_scrub_stamp":"2024-09-13T09:20:22.796093+0000","last_clean_scrub_stamp":"2024-09-13T09:20:22.796093+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:11.050876+0000","scrub_duration":0.0089725240000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"167'657","reported_seq":2743,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:32.156565+0000","last_change":"2024-09-13T09:20:32.156845+0000","last_active":"2024-09-13T09:20:32.156565+0000","last_peered":"2024-09-13T09:20:32.156565+0000","last_clean":"2024-09-13T09:20:32.156565+0000","last_became_active":"2024-09-13T09:18:03.974569+0000","last_became_peered":"2024-09-13T09:18:03.974569+0000","last_unstale":"2024-09-13T09:20:32.156565+0000","last_undegraded":"2024-09-13T09:20:32.156565+0000","last_fullsized":"2024-09-13T09:20:32.156565+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'657","last_scrub_stamp":"2024-09-13T09:20:32.156509+0000","last_deep_scrub":"167'657","last_deep_scrub_stamp":"2024-09-13T09:20:32.156509+0000","last_clean_scrub_stamp":"2024-09-13T09:20:32.156509+0000","objects_scrubbed":7,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:15.684735+0000","scrub_duration":0.015877624,"objects_trimmed":0,"snaptrim_duration":0.0036429610000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4126,"num_read_kb":16863,"num_write":738,"num_write_kb":13412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":741496,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"167'543","reported_seq":2828,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:31.845860+0000","last_change":"2024-09-13T09:20:31.846112+0000","last_active":"2024-09-13T09:20:31.845860+0000","last_peered":"2024-09-13T09:20:31.845860+0000","last_clean":"2024-09-13T09:20:31.845860+0000","last_became_active":"2024-09-13T09:17:27.076185+0000","last_became_peered":"2024-09-13T09:17:27.076185+0000","last_unstale":"2024-09-13T09:20:31.845860+0000","last_undegraded":"2024-09-13T09:20:31.845860+0000","last_fullsized":"2024-09-13T09:20:31.845860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'543","last_scrub_stamp":"2024-09-13T09:20:31.845778+0000","last_deep_scrub":"167'543","last_deep_scrub_stamp":"2024-09-13T09:20:31.845778+0000","last_clean_scrub_stamp":"2024-09-13T09:20:31.845778+0000","objects_scrubbed":4,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:04.926730+0000","scrub_duration":0.010190695,"objects_trimmed":0,"snaptrim_duration":0.00070054600000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":11592,"num_write":623,"num_write_kb":5754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"167'509","reported_seq":3894,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:26.289700+0000","last_change":"2024-09-13T09:20:26.289939+0000","last_active":"2024-09-13T09:20:26.289700+0000","last_peered":"2024-09-13T09:20:26.289700+0000","last_clean":"2024-09-13T09:20:26.289700+0000","last_became_active":"2024-09-13T09:19:34.891064+0000","last_became_peered":"2024-09-13T09:19:34.891064+0000","last_unstale":"2024-09-13T09:20:26.289700+0000","last_undegraded":"2024-09-13T09:20:26.289700+0000","last_fullsized":"2024-09-13T09:20:26.289700+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'509","last_scrub_stamp":"2024-09-13T09:20:26.289647+0000","last_deep_scrub":"167'509","last_deep_scrub_stamp":"2024-09-13T09:20:26.289647+0000","last_clean_scrub_stamp":"2024-09-13T09:20:26.289647+0000","objects_scrubbed":4,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:06:11.262198+0000","scrub_duration":0.010058200999999999,"objects_trimmed":0,"snaptrim_duration":0.00078961199999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":3950,"num_read_kb":15857,"num_write":467,"num_write_kb":7750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":4249683,"num_keys_recovered":143,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"167'432","reported_seq":1851,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:29.229192+0000","last_change":"2024-09-13T09:20:29.229428+0000","last_active":"2024-09-13T09:20:29.229192+0000","last_peered":"2024-09-13T09:20:29.229192+0000","last_clean":"2024-09-13T09:20:29.229192+0000","last_became_active":"2024-09-13T09:17:40.801195+0000","last_became_peered":"2024-09-13T09:17:40.801195+0000","last_unstale":"2024-09-13T09:20:29.229192+0000","last_undegraded":"2024-09-13T09:20:29.229192+0000","last_fullsized":"2024-09-13T09:20:29.229192+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'432","last_scrub_stamp":"2024-09-13T09:20:29.229135+0000","last_deep_scrub":"167'432","last_deep_scrub_stamp":"2024-09-13T09:20:29.229135+0000","last_clean_scrub_stamp":"2024-09-13T09:20:29.229135+0000","objects_scrubbed":3,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:51.153252+0000","scrub_duration":0.0090859649999999997,"objects_trimmed":0,"snaptrim_duration":0.001195911,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2626,"num_read_kb":8389,"num_write":522,"num_write_kb":11747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":67680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"167'688","reported_seq":1704,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:57.868243+0000","last_change":"2024-09-13T09:20:57.868490+0000","last_active":"2024-09-13T09:20:57.868243+0000","last_peered":"2024-09-13T09:20:57.868243+0000","last_clean":"2024-09-13T09:20:57.868243+0000","last_became_active":"2024-09-13T09:19:49.736505+0000","last_became_peered":"2024-09-13T09:19:49.736505+0000","last_unstale":"2024-09-13T09:20:57.868243+0000","last_undegraded":"2024-09-13T09:20:57.868243+0000","last_fullsized":"2024-09-13T09:20:57.868243+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'688","last_scrub_stamp":"2024-09-13T09:20:57.868188+0000","last_deep_scrub":"167'688","last_deep_scrub_stamp":"2024-09-13T09:20:57.868188+0000","last_clean_scrub_stamp":"2024-09-13T09:20:57.868188+0000","objects_scrubbed":5,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:03.691722+0000","scrub_duration":0.011930118999999999,"objects_trimmed":0,"snaptrim_duration":0.00218943,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2689,"num_read_kb":8298,"num_write":946,"num_write_kb":9158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":43,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"167'695","reported_seq":3246,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:30.774226+0000","last_change":"2024-09-13T09:20:30.774467+0000","last_active":"2024-09-13T09:20:30.774226+0000","last_peered":"2024-09-13T09:20:30.774226+0000","last_clean":"2024-09-13T09:20:30.774226+0000","last_became_active":"2024-09-13T09:17:26.713690+0000","last_became_peered":"2024-09-13T09:17:26.713690+0000","last_unstale":"2024-09-13T09:20:30.774226+0000","last_undegraded":"2024-09-13T09:20:30.774226+0000","last_fullsized":"2024-09-13T09:20:30.774226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'695","last_scrub_stamp":"2024-09-13T09:20:30.774168+0000","last_deep_scrub":"167'695","last_deep_scrub_stamp":"2024-09-13T09:20:30.774168+0000","last_clean_scrub_stamp":"2024-09-13T09:20:30.774168+0000","objects_scrubbed":2,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:07.567176+0000","scrub_duration":0.0084296590000000008,"objects_trimmed":0,"snaptrim_duration":0.0035729350000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4417,"num_read_kb":31654,"num_write":734,"num_write_kb":30325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"167'393","reported_seq":1399,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:41.692363+0000","last_change":"2024-09-13T09:20:41.692623+0000","last_active":"2024-09-13T09:20:41.692363+0000","last_peered":"2024-09-13T09:20:41.692363+0000","last_clean":"2024-09-13T09:20:41.692363+0000","last_became_active":"2024-09-13T09:17:26.709524+0000","last_became_peered":"2024-09-13T09:17:26.709524+0000","last_unstale":"2024-09-13T09:20:41.692363+0000","last_undegraded":"2024-09-13T09:20:41.692363+0000","last_fullsized":"2024-09-13T09:20:41.692363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'393","last_scrub_stamp":"2024-09-13T09:20:41.692303+0000","last_deep_scrub":"167'393","last_deep_scrub_stamp":"2024-09-13T09:20:41.692303+0000","last_clean_scrub_stamp":"2024-09-13T09:20:41.692303+0000","objects_scrubbed":10,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:33:22.356565+0000","scrub_duration":0.018884205000000001,"objects_trimmed":0,"snaptrim_duration":0.00068946100000000005,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1301,"num_read_kb":10607,"num_write":406,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"167'984","reported_seq":4069,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:33.731999+0000","last_change":"2024-09-13T09:20:33.732226+0000","last_active":"2024-09-13T09:20:33.731999+0000","last_peered":"2024-09-13T09:20:33.731999+0000","last_clean":"2024-09-13T09:20:33.731999+0000","last_became_active":"2024-09-13T09:17:26.698692+0000","last_became_peered":"2024-09-13T09:17:26.698692+0000","last_unstale":"2024-09-13T09:20:33.731999+0000","last_undegraded":"2024-09-13T09:20:33.731999+0000","last_fullsized":"2024-09-13T09:20:33.731999+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'984","last_scrub_stamp":"2024-09-13T09:20:33.731944+0000","last_deep_scrub":"167'984","last_deep_scrub_stamp":"2024-09-13T09:20:33.731944+0000","last_clean_scrub_stamp":"2024-09-13T09:20:33.731944+0000","objects_scrubbed":4,"log_size":984,"log_dups_size":0,"ondisk_log_size":984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:49.528565+0000","scrub_duration":0.011015433,"objects_trimmed":0,"snaptrim_duration":0.0024735249999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7129,"num_read_kb":22784,"num_write":1446,"num_write_kb":10311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"167'406","reported_seq":2324,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:22.301547+0000","last_change":"2024-09-13T09:20:22.301848+0000","last_active":"2024-09-13T09:20:22.301547+0000","last_peered":"2024-09-13T09:20:22.301547+0000","last_clean":"2024-09-13T09:20:22.301547+0000","last_became_active":"2024-09-13T09:19:20.803495+0000","last_became_peered":"2024-09-13T09:19:20.803495+0000","last_unstale":"2024-09-13T09:20:22.301547+0000","last_undegraded":"2024-09-13T09:20:22.301547+0000","last_fullsized":"2024-09-13T09:20:22.301547+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'406","last_scrub_stamp":"2024-09-13T09:20:22.301493+0000","last_deep_scrub":"167'406","last_deep_scrub_stamp":"2024-09-13T09:20:22.301493+0000","last_clean_scrub_stamp":"2024-09-13T09:20:22.301493+0000","objects_scrubbed":4,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:27.599633+0000","scrub_duration":0.010950427,"objects_trimmed":0,"snaptrim_duration":0.00072899800000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2117,"num_read_kb":10151,"num_write":496,"num_write_kb":12845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":5102507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"167'553","reported_seq":3026,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:29.788358+0000","last_change":"2024-09-13T09:20:29.788575+0000","last_active":"2024-09-13T09:20:29.788358+0000","last_peered":"2024-09-13T09:20:29.788358+0000","last_clean":"2024-09-13T09:20:29.788358+0000","last_became_active":"2024-09-13T09:17:26.709625+0000","last_became_peered":"2024-09-13T09:17:26.709625+0000","last_unstale":"2024-09-13T09:20:29.788358+0000","last_undegraded":"2024-09-13T09:20:29.788358+0000","last_fullsized":"2024-09-13T09:20:29.788358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'553","last_scrub_stamp":"2024-09-13T09:20:29.788300+0000","last_deep_scrub":"167'553","last_deep_scrub_stamp":"2024-09-13T09:20:29.788300+0000","last_clean_scrub_stamp":"2024-09-13T09:20:29.788300+0000","objects_scrubbed":4,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:18:15.140691+0000","scrub_duration":0.010107418,"objects_trimmed":0,"snaptrim_duration":0.0020210359999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4916,"num_read_kb":14560,"num_write":713,"num_write_kb":15391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"167'442","reported_seq":4951,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:22.892931+0000","last_change":"2024-09-13T09:20:22.893154+0000","last_active":"2024-09-13T09:20:22.892931+0000","last_peered":"2024-09-13T09:20:22.892931+0000","last_clean":"2024-09-13T09:20:22.892931+0000","last_became_active":"2024-09-13T09:17:40.789499+0000","last_became_peered":"2024-09-13T09:17:40.789499+0000","last_unstale":"2024-09-13T09:20:22.892931+0000","last_undegraded":"2024-09-13T09:20:22.892931+0000","last_fullsized":"2024-09-13T09:20:22.892931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'442","last_scrub_stamp":"2024-09-13T09:20:22.892876+0000","last_deep_scrub":"167'442","last_deep_scrub_stamp":"2024-09-13T09:20:22.892876+0000","last_clean_scrub_stamp":"2024-09-13T09:20:22.892876+0000","objects_scrubbed":4,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:18.401596+0000","scrub_duration":0.0099034080000000007,"objects_trimmed":0,"snaptrim_duration":0.0010644070000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5803,"num_read_kb":44996,"num_write":398,"num_write_kb":17054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"167'424","reported_seq":3635,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T09:20:44.657125+0000","last_change":"2024-09-13T09:20:44.657347+0000","last_active":"2024-09-13T09:20:44.657125+0000","last_peered":"2024-09-13T09:20:44.657125+0000","last_clean":"2024-09-13T09:20:44.657125+0000","last_became_active":"2024-09-13T09:17:26.700786+0000","last_became_peered":"2024-09-13T09:17:26.700786+0000","last_unstale":"2024-09-13T09:20:44.657125+0000","last_undegraded":"2024-09-13T09:20:44.657125+0000","last_fullsized":"2024-09-13T09:20:44.657125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"167'424","last_scrub_stamp":"2024-09-13T09:20:44.657066+0000","last_deep_scrub":"167'424","last_deep_scrub_stamp":"2024-09-13T09:20:44.657066+0000","last_clean_scrub_stamp":"2024-09-13T09:20:44.657066+0000","objects_scrubbed":3,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:37:38.394502+0000","scrub_duration":0.0089945570000000002,"objects_trimmed":0,"snaptrim_duration":0.003077476,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4122,"num_read_kb":18425,"num_write":408,"num_write_kb":9869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":152,"num_read":126915,"num_read_kb":555724,"num_write":19133,"num_write_kb":379149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1469,"num_bytes_recovered":28693609,"num_keys_recovered":250,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16840,"ondisk_log_size":16840,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":30095,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59372,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312468,"statfs":{"total":96636764160,"available":96575967232,"internally_reserved":0,"allocated":2068480,"data_stored":1548192,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.95699999999999996,"15min":0.89200000000000002},"min":{"1min":0.63500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.1799999999999999,"5min":11.683,"15min":11.683},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.0269999999999999,"15min":0.98099999999999998},"min":{"1min":0.60699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2729999999999999,"5min":11.843999999999999,"15min":11.843999999999999},"last":0.60999999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.85699999999999998,"15min":0.86299999999999999},"min":{"1min":0.54900000000000004,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.175,"5min":1.286,"15min":1.286},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.95099999999999996,"15min":0.98299999999999998},"min":{"1min":0.61699999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.304,"5min":3.9039999999999999,"15min":3.9039999999999999},"last":0.81999999999999995}]},{"osd":2,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.82099999999999995,"15min":0.81000000000000005},"min":{"1min":0.53800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.876},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.73199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.996}]},{"osd":3,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.0289999999999999,"15min":0.97099999999999997},"min":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.597,"5min":12.012,"15min":12.012},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.139,"15min":1.077},"min":{"1min":0.66400000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.615,"5min":11.776999999999999,"15min":11.776999999999999},"last":0.91600000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.0049999999999999,"15min":1.0529999999999999},"min":{"1min":0.55700000000000005,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4319999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95799999999999996,"15min":0.96899999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.079,"15min":1.073},"min":{"1min":0.59799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6990000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1160000000000001,"15min":1.143},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.651,"5min":3.125,"15min":3.125},"last":1.028}]},{"osd":6,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.163,"15min":1.222},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.9910000000000001,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.117,"15min":1.1220000000000001},"min":{"1min":0.52000000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4750000000000001,"5min":1.996,"15min":1.996},"last":0.95199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41268,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330572,"statfs":{"total":96636764160,"available":96594505728,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.438,"5min":0.314,"15min":0.314},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.877},"min":{"1min":0.40899999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3580000000000001,"5min":2.222,"15min":2.222},"last":0.68200000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.83399999999999996,"15min":0.81200000000000006},"min":{"1min":0.50800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.98199999999999998,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.607,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.1080000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.65700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4319999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.98499999999999999,"15min":1.0029999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.613,"5min":2.508,"15min":2.508},"last":1.161}]},{"osd":3,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.66900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.55,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.083},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.2450000000000001,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.85799999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.95199999999999996,"15min":0.97399999999999998},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2669999999999999,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.008,"15min":1},"min":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5800000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.92300000000000004,"15min":0.90000000000000002},"min":{"1min":0.58999999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.57599999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.8120000000000001,"5min":2.286,"15min":2.286},"last":1.0069999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:20:09 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.093,"15min":1.087},"min":{"1min":0.621,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.611,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.079,"15min":1.0489999999999999},"min":{"1min":0.69899999999999995,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.129}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55224,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316616,"statfs":{"total":96636764160,"available":96580214784,"internally_reserved":0,"allocated":2015232,"data_stored":957843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":4.992,"15min":6.8739999999999997},"min":{"1min":0.47299999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.306,"5min":132.32900000000001,"15min":132.32900000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81499999999999995,"15min":0.81699999999999995},"min":{"1min":0.50900000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.151,"5min":1.335,"15min":1.335},"last":1.0269999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.84099999999999997,"15min":0.877},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2010000000000001,"5min":1.946,"15min":1.946},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97899999999999998,"15min":1.0149999999999999},"min":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4099999999999999,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84199999999999997,"15min":0.84399999999999997},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.875,"15min":0.89300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2130000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.0549999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.94399999999999995},"min":{"1min":0.61299999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.286,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.98399999999999999,"15min":1},"min":{"1min":0.60899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.476,"15min":1.476},"last":0.97899999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.88900000000000001,"15min":0.872},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.458,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.002},{"interface":"front","average":{"1min":1.103,"5min":5.375,"15min":7.2999999999999998},"min":{"1min":0.55100000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.395,"5min":132.23599999999999,"15min":132.23599999999999},"last":1.1910000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.97099999999999997,"15min":0.96699999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.997,"5min":1.0069999999999999,"15min":1.03},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.5820000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:20:15 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.085,"15min":1.0780000000000001},"min":{"1min":0.67900000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.359,"5min":1.655,"15min":1.655},"last":1.143},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.115,"15min":1.131},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.4850000000000001,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.1100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51128,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320712,"statfs":{"total":96636764160,"available":96584409088,"internally_reserved":0,"allocated":2015232,"data_stored":957843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3560000000000001,"5min":1.357,"15min":1.357},"last":1.155},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.94599999999999995,"15min":0.94799999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.1910000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91400000000000003,"15min":0.92600000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.81100000000000005,"15min":0.81999999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.88100000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.877},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.006,"5min":0.88,"15min":0.84299999999999997},"min":{"1min":0.56999999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.91900000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86899999999999999,"15min":0.85399999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3300000000000001,"5min":1.575,"15min":1.575},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.069,"5min":1.0249999999999999,"15min":0.97799999999999998},"min":{"1min":0.628,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0899999999999999,"5min":4.7389999999999999,"15min":4.7389999999999999},"last":0.97799999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.92000000000000004,"15min":0.89600000000000002},"min":{"1min":0.45400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0329999999999999},"min":{"1min":0.53500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.181,"5min":1.077,"15min":1.0629999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.4119999999999999,"5min":2.4119999999999999,"15min":2.4119999999999999},"last":0.74399999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.034,"15min":1.026},"min":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5609999999999999,"5min":1.597,"15min":1.597},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.125,"15min":1.131},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.075}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77364,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94294476,"statfs":{"total":96636764160,"available":96557543424,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.249,"5min":0.215,"15min":0.215},"max":{"1min":4.492,"5min":4.492,"15min":4.492},"last":0.498},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.33100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.337,"5min":1.625,"15min":1.625},"last":0.66000000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.38600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.4990000000000001,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.72099999999999997,"15min":0.68899999999999995},"min":{"1min":0.29299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.3890000000000002,"5min":4.3890000000000002,"15min":4.3890000000000002},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.73099999999999998,"15min":0.70299999999999996},"min":{"1min":0.44,"5min":0.312,"15min":0.312},"max":{"1min":1.45,"5min":1.544,"15min":1.544},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.79600000000000004,"15min":0.77600000000000002},"min":{"1min":0.41699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.2880000000000003,"5min":4.2880000000000003,"15min":4.2880000000000003},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.90400000000000003,"15min":0.91000000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.161},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.94899999999999995,"15min":0.94699999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0049999999999999,"15min":0.98299999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":5.3319999999999999,"5min":5.3319999999999999,"15min":5.3319999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.93899999999999995,"15min":0.93700000000000006},"min":{"1min":0.55200000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.196}]},{"osd":6,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.022,"15min":1.0289999999999999},"min":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.238},{"interface":"front","average":{"1min":1.272,"5min":1.1200000000000001,"15min":1.1080000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":5.25,"5min":5.25,"15min":5.25},"last":0.92300000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.0149999999999999,"15min":1.012},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.6899999999999999,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.5269999999999999,"15min":1.726},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.504},"max":{"1min":2.097,"5min":15.4,"15min":15.4},"last":1.085}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41268,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330572,"statfs":{"total":96636764160,"available":96594505728,"internally_reserved":0,"allocated":2011136,"data_stored":957824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.621},"min":{"1min":0.247,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1899999999999999,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.72499999999999998,"15min":0.70499999999999996},"min":{"1min":0.28399999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.671,"5min":2.8490000000000002,"15min":2.8490000000000002},"last":0.89400000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.79000000000000004,"15min":0.79300000000000004},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.337,"5min":1.907,"15min":1.907},"last":1.069}]},{"osd":3,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.77600000000000002,"15min":0.74399999999999999},"min":{"1min":0.48999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.256,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.83899999999999997,"15min":0.871},"min":{"1min":0.34000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.47,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.86499999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.97399999999999998,"15min":0.96999999999999997},"min":{"1min":0.76200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4470000000000001,"5min":1.782,"15min":1.782},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.83499999999999996,"15min":0.81699999999999995},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.97999999999999998,"15min":0.96799999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.252},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0329999999999999,"15min":1.0289999999999999},"min":{"1min":0.57199999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5580000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2949999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.55600000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.012,"15min":1.0049999999999999},"min":{"1min":0.623,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.923,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.012}]},{"osd":7,"last update":"Fri Sep 13 09:20:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.121,"15min":1.133},"min":{"1min":0.68899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.573,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.0049999999999999,"15min":1.0029999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5620000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.1699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":2019328,"data_stored":957862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.623},"min":{"1min":0.35799999999999998,"5min":0.255,"15min":0.255},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.34300000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.95499999999999996,"5min":1.484,"15min":1.484},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72899999999999998,"15min":0.75700000000000001},"min":{"1min":0.36699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.0569999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.753},"min":{"1min":0.44600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.135,"5min":1.264,"15min":1.264},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76500000000000001,"15min":0.76900000000000002},"min":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1499999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.135}]},{"osd":4,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3080000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97699999999999998,"15min":1.0109999999999999},"min":{"1min":0.64800000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3620000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.0760000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0780000000000001,"15min":1.0409999999999999},"min":{"1min":0.60799999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.524,"5min":8.3460000000000001,"15min":8.3460000000000001},"last":1.216},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98799999999999999,"15min":1.018},"min":{"1min":0.56599999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4710000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.95499999999999996}]},{"osd":6,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.01,"15min":1.044},"min":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.573,"5min":1.647,"15min":1.647},"last":0.87},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.002,"15min":1.028},"min":{"1min":0.61499999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1}]},{"osd":7,"last update":"Fri Sep 13 09:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.024,"15min":1.05},"min":{"1min":0.69099999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5800000000000001,"5min":1.629,"15min":1.629},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.081,"15min":1.1140000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.0309999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33116,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":2052096,"data_stored":1548192,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.316,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.325,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.625,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.006,"5min":1.284,"15min":1.284},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92600000000000005,"5min":1.5,"15min":1.5},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.39000000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.129,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.88200000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.72499999999999998,"15min":0.69899999999999995},"min":{"1min":0.42699999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.452,"5min":2.242,"15min":2.242},"last":1.095},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.78900000000000003},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2250000000000001,"5min":1.528,"15min":1.528},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.89200000000000002,"15min":0.877},"min":{"1min":0.53700000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0309999999999999,"15min":0.96599999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.484,"5min":9.5649999999999995,"15min":9.5649999999999995},"last":0.92800000000000005}]},{"osd":5,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0469999999999999,"15min":0.95799999999999996},"min":{"1min":0.67900000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.375,"5min":9.6120000000000001,"15min":9.6120000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.419,"5min":1.476,"15min":1.476},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0620000000000001,"15min":1.085},"min":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4279999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.996,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.482,"5min":1.516,"15min":1.516},"last":1.1319999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:20:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0229999999999999,"15min":1.0309999999999999},"min":{"1min":0.58799999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.5669999999999999,"5min":1.617,"15min":1.617},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.995,"5min":1.1379999999999999,"15min":1.0820000000000001},"min":{"1min":0.67300000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.546,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":1.016}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":10378,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9339,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:21:03.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T09:21:03.406 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T09:21:03.407 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T09:21:03.407 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T09:21:03.407 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:03.498 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T09:21:03.498 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T09:21:03.499 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:03.573 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T09:21:03.573 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T09:21:03.573 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:09.642 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-13T09:21:09.675 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T09:21:09.676 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T09:21:09.676 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:09.745 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T09:21:09.746 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T09:21:09.746 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:09.830 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T09:21:09.830 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T09:21:09.830 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:09.909 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T09:21:09.909 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T09:21:09.909 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:09.996 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T09:21:09.996 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T09:21:09.996 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:15.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-13T09:21:15.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-13T09:21:16.098 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T09:21:16.098 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T09:21:16.098 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T09:21:16.098 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:16.128 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T09:21:16.129 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T09:21:16.129 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:16.176 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T09:21:16.176 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T09:21:16.176 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T09:21:16.176 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:16.190 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T09:21:16.191 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T09:21:16.191 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:16.255 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T09:21:16.255 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T09:21:16.256 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:21:16.281 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T09:21:16.281 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T09:21:16.281 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-13T09:21:16.316 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-13T09:21:16.316 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T09:21:16.590 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-13T09:21:16.591 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T09:21:16.920 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-13T09:21:16.920 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T09:21:17.066 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-13T09:21:17.067 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T09:21:17.211 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-13T09:21:17.212 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T09:21:17.353 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-13T09:21:17.353 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T09:21:17.666 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-13T09:21:17.666 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T09:21:17.814 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-13T09:21:17.815 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T09:21:17.933 INFO:tasks.ceph:Archiving mon data... 2024-09-13T09:21:17.934 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022/data/mon.a.tgz 2024-09-13T09:21:17.934 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-09-13T09:21:17.940 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.ztYRY00yIq 2024-09-13T09:21:17.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ztYRY00yIq 2024-09-13T09:21:18.093 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.ztYRY00yIq 2024-09-13T09:21:18.156 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.ztYRY00yIq is 393KB 2024-09-13T09:21:18.222 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.ztYRY00yIq 2024-09-13T09:21:18.227 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022/data/mon.c.tgz 2024-09-13T09:21:18.227 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-09-13T09:21:18.280 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.9Z25LbesHA 2024-09-13T09:21:18.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.9Z25LbesHA 2024-09-13T09:21:18.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.9Z25LbesHA 2024-09-13T09:21:18.529 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.9Z25LbesHA is 533KB 2024-09-13T09:21:18.630 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.9Z25LbesHA 2024-09-13T09:21:18.635 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022/data/mon.b.tgz 2024-09-13T09:21:18.636 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-09-13T09:21:18.641 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.fKkHRvixdZ 2024-09-13T09:21:18.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.fKkHRvixdZ 2024-09-13T09:21:18.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.fKkHRvixdZ 2024-09-13T09:21:18.876 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.fKkHRvixdZ is 532KB 2024-09-13T09:21:18.941 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.fKkHRvixdZ 2024-09-13T09:21:18.946 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T09:21:18.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T09:21:18.949 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T09:21:19.000 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T09:21:19.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T09:21:19.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T09:21:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T09:21:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T09:21:19.062 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T09:21:19.063 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022/remote/smithi032/crash 2024-09-13T09:21:19.064 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T09:21:19.081 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022/remote/smithi046/crash 2024-09-13T09:21:19.082 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T09:21:19.121 INFO:tasks.ceph:Compressing logs... 2024-09-13T09:21:19.121 DEBUG:teuthology.orchestra.run.smithi032:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:21:19.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:21:20.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-13T09:21:27.071 INFO:tasks.ceph:Archiving logs... 2024-09-13T09:21:27.072 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022/remote/smithi032/log 2024-09-13T09:21:27.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T09:21:27.921 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022/remote/smithi046/log 2024-09-13T09:21:27.922 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T09:21:28.577 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T09:21:28.586 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T09:21:28.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T09:21:28.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T09:21:28.661 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T09:21:28.669 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T09:21:28.669 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:21:28.672 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:21:28.683 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T09:21:28.684 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2024-09-13T09:21:28.684 INFO:teuthology.orchestra.run.smithi032.stdout:+hv01.front.sepi 67.205.162.81 3 u 8 64 177 0.096 -1.095 0.871 2024-09-13T09:21:28.684 INFO:teuthology.orchestra.run.smithi032.stdout:*hv02.front.sepi 96.245.170.99 3 u 11 64 177 0.143 -0.418 0.990 2024-09-13T09:21:28.684 INFO:teuthology.orchestra.run.smithi032.stdout:+hv03.front.sepi 74.6.168.72 3 u 6 64 177 0.089 -0.538 0.962 2024-09-13T09:21:28.684 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:21:28.685 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T09:21:28.686 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2024-09-13T09:21:28.686 INFO:teuthology.orchestra.run.smithi046.stdout:+hv01.front.sepi 67.205.162.81 3 u 9 64 177 0.099 -0.690 0.249 2024-09-13T09:21:28.686 INFO:teuthology.orchestra.run.smithi046.stdout:+hv02.front.sepi 96.245.170.99 3 u 13 64 177 0.081 -0.324 0.256 2024-09-13T09:21:28.686 INFO:teuthology.orchestra.run.smithi046.stdout:*hv03.front.sepi 74.6.168.72 3 u 8 64 177 0.087 -0.648 0.335 2024-09-13T09:21:28.686 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:21:28.687 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T09:21:28.695 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T09:21:28.696 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T09:21:28.704 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T09:21:28.713 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T09:21:28.721 INFO:teuthology.task.internal:Duration was 871.419811 seconds 2024-09-13T09:21:28.721 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T09:21:28.729 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T09:21:28.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T09:21:28.732 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T09:21:28.770 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T09:21:28.770 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2024-09-13T09:21:28.770 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T09:21:28.825 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-09-13T09:21:28.825 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T09:21:28.836 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T09:21:28.836 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:21:28.869 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:21:28.906 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T09:21:28.906 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T09:21:28.940 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T09:21:29.018 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T09:21:29.028 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T09:21:29.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T09:21:29.040 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T09:21:29.053 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T09:21:29.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:21:29.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:21:29.094 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2024-09-13T09:21:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-09-13T09:21:29.132 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:21:29.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:21:29.167 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:21:29.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:21:29.177 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T09:21:29.187 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T09:21:29.188 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022/remote/smithi032 2024-09-13T09:21:29.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T09:21:29.239 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901022/remote/smithi046 2024-09-13T09:21:29.239 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T09:21:29.259 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T09:21:29.259 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T09:21:29.273 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T09:21:29.302 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T09:21:29.312 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T09:21:29.312 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T09:21:29.320 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T09:21:29.321 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T09:21:29.323 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T09:21:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: 658467 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 09:21 /home/ubuntu/cephtest 2024-09-13T09:21:29.350 INFO:teuthology.orchestra.run.smithi046.stdout: 658625 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 09:21 /home/ubuntu/cephtest 2024-09-13T09:21:29.351 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T09:21:29.361 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T09:21:29.383 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough} duration: 871.4198114871979 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-13T09:21:29.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T09:21:29.463 INFO:teuthology.run:pass